Jul 25 00:36:16.299825 ip-10-0-140-134 systemd[1]: Starting Kubernetes Kubelet...
Jul 25 00:36:16.672028 ip-10-0-140-134 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 00:36:16.672028 ip-10-0-140-134 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 25 00:36:16.672028 ip-10-0-140-134 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 00:36:16.672028 ip-10-0-140-134 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 25 00:36:16.672028 ip-10-0-140-134 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 00:36:16.672941 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.672829    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 25 00:36:16.677943 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677929    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:36:16.677943 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677942    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677947    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677950    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677953    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677956    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677959    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677962    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677964    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677968    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677971    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677973    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677976    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677978    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677981    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677983    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677986    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677989    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677993    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.677997    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678000    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:36:16.678001 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678004    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678006    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678009    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678012    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678015    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678018    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678021    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678023    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678025    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678028    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678030    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678033    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678035    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678037    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678040    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678042    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678045    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678048    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678051    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:36:16.678469 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678053    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678056    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678058    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678061    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678063    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678065    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678068    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678070    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678073    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678075    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678080    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678083    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678087    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678089    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678092    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678097    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678100    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678102    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678105    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678107    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:36:16.678923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678110    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678112    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678115    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678117    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678120    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678122    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678125    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678127    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678129    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678133    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678137    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678141    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678144    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678146    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678149    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678152    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678154    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678157    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678160    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:36:16.679433 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678162    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678164    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678167    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678169    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678172    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678174    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678177    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678179    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678182    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678185    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678188    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678190    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678193    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678195    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678198    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678200    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678202    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678205    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678208    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678210    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:36:16.679884 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678914    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678921    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678925    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678928    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678932    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678935    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678938    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678941    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678944    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678948    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678951    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678954    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678957    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678959    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678962    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678964    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678967    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678969    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678972    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678975    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:36:16.680375 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678978    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678981    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678984    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678986    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678989    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678991    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678994    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678997    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.678999    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679002    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679004    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679007    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679009    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679012    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679014    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679016    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679019    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679021    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679024    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679026    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:36:16.680882 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679030    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679032    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679035    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679037    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679040    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679042    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679045    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679047    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679050    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679054    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679057    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679062    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679065    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679067    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679070    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679073    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679076    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679078    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679081    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679083    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:36:16.681370 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679086    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679088    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679090    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679093    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679095    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679098    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679100    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679102    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679105    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679109    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679111    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679114    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679117    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679119    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679122    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679124    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679128    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679131    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679133    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679136    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:36:16.681861 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679138    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679141    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679143    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679146    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679149    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679151    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679154    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679156    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679159    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679161    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679163    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679166    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679168    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679171    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679174    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679178    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679181    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679183    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679186    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:36:16.682334 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679587    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679593    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679596    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679599    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679606    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679609    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679612    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679615    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679617    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679623    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679625    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679628    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679630    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679633    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679636    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679639    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679641    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679644    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679646    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679649    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:36:16.682790 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679651    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679653    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679657    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679661    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679664    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679667    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679669    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679672    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679675    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679677    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679679    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679682    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679684    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679687    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679689    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679691    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679694    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679696    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679699    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679702    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:36:16.683254 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679704    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679708    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679710    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679713    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679715    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679718    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679721    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679723    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679726    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679728    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679731    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679733    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679735    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679738    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679740    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679743    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679745    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679747    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679750    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679752    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:36:16.683733 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679755    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679758    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679760    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679762    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679765    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679767    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679770    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679774    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679778    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679781    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679784    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679787    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679789    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679793    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679796    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679799    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679801    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679804    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679807    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:36:16.684259 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679810    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679812    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679815    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679817    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679820    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679822    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679825    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679827    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679830    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679832    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679837    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679840    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679843    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679845    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679848    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679851    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679854    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679856    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679859    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:36:16.684747 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.679863    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679923    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679930    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679937    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679941    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679945    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679948    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679953    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679957    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679960    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679963    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679967    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679970    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679973    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679976    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679979    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679982    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679985    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679988    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679992    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679994    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.679997    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680000    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680004    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 25 00:36:16.685324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680007    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680010    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680013    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680022    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680026    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680029    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680032    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680035    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680039    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680043    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680046    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680049    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680051    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680073    2566 flags.go:64] FLAG: --enable-server="true"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680078    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680086    2566 flags.go:64] FLAG: --event-burst="100"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680089    2566 flags.go:64] FLAG: --event-qps="50"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680093    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680096    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680099    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680102    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680105    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680108    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680111    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680114    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 25 00:36:16.685906 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680117    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680120    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680123    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680126    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680129    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680131    2566 flags.go:64] FLAG: --feature-gates=""
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680135    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680138    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680141    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680144    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680147    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680150    2566 flags.go:64] FLAG: --help="false"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680153    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-140-134.ec2.internal"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680156    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680159    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680163    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680166    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680170    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680172    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680175    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680178    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680181    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680186    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680189    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 25 00:36:16.686507 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680192    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680195    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680197    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680200    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680203    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680206    2566 flags.go:64] FLAG: --lock-file=""
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680209    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680212    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680215    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680220    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680223    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680226    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680229    2566 flags.go:64] FLAG: --logging-format="text"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680231    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680235    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680238    2566 flags.go:64] FLAG: --manifest-url=""
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680240    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680244    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680247    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680251    2566 flags.go:64] FLAG: --max-pods="110"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680254    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680257    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680260    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680264    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680267    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 25 00:36:16.687127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680270    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680273    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680280    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680282    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680286    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680290    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680294    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680299    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680302    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680305    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680308    2566 flags.go:64] FLAG: --port="10250"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680312    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680315    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d75e4423d48e6729"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680318    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680321    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680324    2566 flags.go:64] FLAG: --register-node="true"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680327    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680330    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680333    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680336    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680338    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680342    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680346    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680349    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680352    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 25 00:36:16.687770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680354    2566 flags.go:64] FLAG: --runonce="false"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680357    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680360    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680363    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680366    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680369    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680373    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680376    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680379    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680382    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680385    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680389    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680393    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680396    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680399    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680402    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680408    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680410    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680413    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680417    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680420    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680423    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680426    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680428    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680431    2566 flags.go:64] FLAG: --v="2"
Jul 25 00:36:16.688409 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680435    2566 flags.go:64] FLAG: --version="false"
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680439    2566 flags.go:64] FLAG: --vmodule=""
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680443    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.680446    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680570    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680574    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680578    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680581    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680584    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680586    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680589    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680592    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680594    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680598    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680601    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680603    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680606    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680609    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680611    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680614    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:36:16.689029 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680617    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680619    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680623    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680625    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680628    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680632    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680635    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680638    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680641    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680644    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680646    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680648    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680651    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680654    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680656    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680659    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680661    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680664    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680666    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:36:16.689522 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680669    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680671    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680674    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680676    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680679    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680681    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680685    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680687    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680690    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680692    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680695    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680698    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680700    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680703    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680705    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680708    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680711    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680714    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680717    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680719    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:36:16.689992 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680722    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680724    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680727    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680729    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680731    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680734    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680736    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680739    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680741    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680759    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680763    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680766    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680770    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680774    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680777    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680780    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680783    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680786    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680791    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680794    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:36:16.690459 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680796    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680799    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680802    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680804    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680807    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680810    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680813    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680816    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680819    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680822    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680824    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680827    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680830    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680832    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680835    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680837    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680840    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680842    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680845    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680849    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:36:16.690941 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680852    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680857    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680860    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.680862    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.681452    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.688596    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.688614    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.688632    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.688638    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688700    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688705    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688708    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688711    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688714    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:36:16.691406 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688718    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688723    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688726    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688729    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688731    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688734    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688736    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688739    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688742    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688744    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688747    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688750    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688752    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688755    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688757    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688760    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688762    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688765    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688767    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688771    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:36:16.691833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688775    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688778    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688781    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688784    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688793    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688796    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688799    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688802    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688804    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688807    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688809    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688812    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688815    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688818    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688821    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688824    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688826    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688829    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688831    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688834    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:36:16.692345 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688836    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688839    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688841    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688843    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688846    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688849    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688851    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688854    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688857    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688859    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688862    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688864    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688867    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688870    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688872    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688875    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688878    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688886    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688889    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:36:16.692836 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688892    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688895    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688897    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688900    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688903    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688905    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688908    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688910    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688913    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688915    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688918    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688921    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688923    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688926    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688928    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688931    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688933    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688936    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688939    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688941    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:36:16.693380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688944    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688946    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688948    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688951    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688953    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688956    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688959    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688961    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688964    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688967    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688970    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688978    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688981    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688983    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.688986    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:36:16.693872 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.688991    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689141    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689147    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689150    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689158    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689161    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689163    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689166    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689169    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689171    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689174    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689176    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689179    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689181    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689184    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689186    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689189    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689191    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689194    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689196    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:36:16.694231 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689199    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689201    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689204    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689206    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689209    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689211    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689214    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689217    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689220    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689233    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689236    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689239    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689241    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689244    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689247    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689249    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689264    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689268    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689271    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:36:16.694717 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689273    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689276    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689278    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689280    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689283    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689285    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689288    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689290    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689293    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689295    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689299    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689302    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689305    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689307    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689310    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689313    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689315    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689318    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689321    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:36:16.695428 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689323    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689326    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689329    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689332    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689340    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689343    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689345    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689348    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689350    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689353    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689355    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689357    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689360    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689362    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689365    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689367    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689370    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689372    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689374    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689377    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:36:16.696234 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689379    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689382    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689387    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689390    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689392    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689395    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689397    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689399    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689402    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689404    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689407    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689409    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689411    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689414    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689417    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689419    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689423    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689432    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689435    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:36:16.696902 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689438    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:36:16.697411 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689441    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:36:16.697411 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:16.689443    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:36:16.697411 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.689448    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 00:36:16.697411 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.689583    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 25 00:36:16.697411 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.693185    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 25 00:36:16.697411 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.694698    2566 server.go:1019] "Starting client certificate rotation"
Jul 25 00:36:16.697411 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.694825    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 25 00:36:16.697411 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.694936    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 25 00:36:16.697411 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.696951    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 25 00:36:16.703053 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.703032    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 25 00:36:16.718784 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.718765    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 25 00:36:16.720516 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.720491    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 25 00:36:16.720873 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.720860    2566 log.go:25] "Validated CRI v1 image API"
Jul 25 00:36:16.722540 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.722524    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 25 00:36:16.725622 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.725595    2566 fs.go:135] Filesystem UUIDs: map[15ab711f-7f1c-4a88-8e27-11a1a1b4d965:/dev/nvme0n1p3 188f8fcc-67c1-4e6b-a2a6-295d2c69706f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 25 00:36:16.725679 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.725621    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 25 00:36:16.732359 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.732250    2566 manager.go:216] Machine: {Timestamp:2026-07-25 00:36:16.729164802 +0000 UTC m=+0.319200657 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3206194 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2448472313f09b731bb382003b5338 SystemUUID:ec244847-2313-f09b-731b-b382003b5338 BootID:d5737f5d-8a6f-405e-9f0d-ac2bdbaf0d85 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e3:0e:0a:ec:69 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e3:0e:0a:ec:69 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:24:a6:d7:c0:5f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 25 00:36:16.732359 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.732354    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 25 00:36:16.732457 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.732430    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 25 00:36:16.735807 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.735784    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 25 00:36:16.735974 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.735810    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-134.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 25 00:36:16.737070 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.737060    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 25 00:36:16.737104 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.737073    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 25 00:36:16.737104 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.737081    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 25 00:36:16.737739 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.737729    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 25 00:36:16.738447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.738438    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 25 00:36:16.740715 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.740703    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 25 00:36:16.740824 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.740814    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 25 00:36:16.740860 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.740844    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 25 00:36:16.740860 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.740855    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 25 00:36:16.740937 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.740871    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 25 00:36:16.740937 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.740883    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 25 00:36:16.740937 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.740891    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 25 00:36:16.741266 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.741253    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 25 00:36:16.741313 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.741272    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 25 00:36:16.743374 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.743354    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 25 00:36:16.745395 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.745381    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 25 00:36:16.745462 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.745411    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 25 00:36:16.746885 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746873    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746889    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746895    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746901    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746909    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746915    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746922    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746927    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746935    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746943    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746953    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 25 00:36:16.746975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.746958    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 25 00:36:16.747963 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.747946    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kbh7b"
Jul 25 00:36:16.748383 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.748370    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 25 00:36:16.748383 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.748382    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 25 00:36:16.751401 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.751384    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-134.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 25 00:36:16.751539 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.751527    2566 server.go:1268] "Started kubelet"
Jul 25 00:36:16.754520 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:16.754412    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 25 00:36:16.754520 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:16.754452    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 25 00:36:16.754775 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.754631    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 25 00:36:16.754897 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.754608    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 25 00:36:16.754965 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.754918    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 25 00:36:16.755024 ip-10-0-140-134 systemd[1]: Started Kubernetes Kubelet.
Jul 25 00:36:16.756161 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.756029    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kbh7b"
Jul 25 00:36:16.757105 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.757082    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 25 00:36:16.757411 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.757396    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 25 00:36:16.760142 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:16.760116    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 25 00:36:16.762085 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:16.761160    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-134.ec2.internal.18c561451abd0988  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-134.ec2.internal,UID:ip-10-0-140-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-134.ec2.internal,},FirstTimestamp:2026-07-25 00:36:16.751503752 +0000 UTC m=+0.341539607,LastTimestamp:2026-07-25 00:36:16.751503752 +0000 UTC m=+0.341539607,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-134.ec2.internal,}"
Jul 25 00:36:16.764658 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.764641    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 25 00:36:16.764658 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.764645    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 25 00:36:16.764802 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:16.764784    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:16.765490 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765474    2566 factory.go:55] Registering systemd factory
Jul 25 00:36:16.765490 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765492    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 25 00:36:16.765638 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765509    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 25 00:36:16.765638 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765522    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 25 00:36:16.765638 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765507    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 25 00:36:16.765752 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765643    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 25 00:36:16.765752 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765651    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 25 00:36:16.765752 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765747    2566 factory.go:153] Registering CRI-O factory
Jul 25 00:36:16.765883 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765761    2566 factory.go:223] Registration of the crio container factory successfully
Jul 25 00:36:16.765883 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765807    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 25 00:36:16.765883 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765825    2566 factory.go:103] Registering Raw factory
Jul 25 00:36:16.765883 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.765835    2566 manager.go:1196] Started watching for new ooms in manager
Jul 25 00:36:16.766141 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.766128    2566 manager.go:349] Starting recovery of all containers
Jul 25 00:36:16.770272 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.770255    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:36:16.773537 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:16.773514    2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-134.ec2.internal\" not found" node="ip-10-0-140-134.ec2.internal"
Jul 25 00:36:16.776407 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.776388    2566 manager.go:354] Recovery completed
Jul 25 00:36:16.780905 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.780892    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:36:16.783725 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.783605    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:36:16.783725 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.783636    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:36:16.783725 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.783649    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:36:16.784217 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.784203    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 25 00:36:16.784217 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.784215    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 25 00:36:16.784319 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.784225    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:36:16.784319 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.784231    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 25 00:36:16.784978 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.784964    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:36:16.785036 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.784990    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:36:16.785036 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.785000    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:36:16.785866 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.785851    2566 policy_none.go:49] "None policy: Start"
Jul 25 00:36:16.785910 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.785873    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 25 00:36:16.785910 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.785887    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 25 00:36:16.786977 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.786966    2566 policy_none.go:47] "Start"
Jul 25 00:36:16.834284 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.834123    2566 manager.go:337] "Starting Device Plugin manager"
Jul 25 00:36:16.834390 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:16.834320    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 25 00:36:16.834390 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.834331    2566 server.go:87] "Starting device plugin registration server"
Jul 25 00:36:16.834594 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.834583    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 25 00:36:16.834631 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.834596    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 25 00:36:16.834723 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.834705    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 25 00:36:16.834834 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.834794    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 25 00:36:16.834834 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.834803    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 25 00:36:16.835380 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:16.835361    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 25 00:36:16.835456 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:16.835411    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:16.934792 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.934729    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:36:16.935804 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.935788    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:36:16.935858 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.935819    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:36:16.935858 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.935830    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:36:16.935858 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.935856    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-134.ec2.internal"
Jul 25 00:36:16.945007 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:16.944986    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-134.ec2.internal"
Jul 25 00:36:16.945069 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:16.945011    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-134.ec2.internal\": node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:18.164029 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.164001    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 25 00:36:18.165246 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.165231    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 25 00:36:18.165299 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.165267    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 25 00:36:18.165299 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.165288    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 25 00:36:18.165382 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.165328    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 25 00:36:18.167644 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.167623    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:36:18.193992 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.193974    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:18.266117 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.266084    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"]
Jul 25 00:36:18.266199 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.266165    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:36:18.267955 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.267934    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:36:18.268020 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.267972    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:36:18.268020 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.267987    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:36:18.268160 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.268142    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/636eedb83704a28c493136e55b005566-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"636eedb83704a28c493136e55b005566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.268198 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.268167    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/636eedb83704a28c493136e55b005566-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"636eedb83704a28c493136e55b005566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.269156 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.269144    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:36:18.269285 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.269270    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.269320 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.269302    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:36:18.269909 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.269888    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:36:18.269909 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.269907    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:36:18.270047 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.269918    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:36:18.270047 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.269929    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:36:18.270047 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.269949    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:36:18.270047 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.269963    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:36:18.271236 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.271223    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.271300 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.271246    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:36:18.271860 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.271845    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:36:18.271935 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.271870    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:36:18.271935 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.271885    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:36:18.294142 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.294116    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:18.299847 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.299825    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-134.ec2.internal\" not found" node="ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.304245 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.304229    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-134.ec2.internal\" not found" node="ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.369021 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.368991    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/636eedb83704a28c493136e55b005566-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"636eedb83704a28c493136e55b005566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.369021 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.369024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/636eedb83704a28c493136e55b005566-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"636eedb83704a28c493136e55b005566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.369192 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.369051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/22634a54757c03b5d1c07ddcc38ba334-config\") pod \"kube-apiserver-proxy-ip-10-0-140-134.ec2.internal\" (UID: \"22634a54757c03b5d1c07ddcc38ba334\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.369192 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.369119    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/636eedb83704a28c493136e55b005566-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"636eedb83704a28c493136e55b005566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.369192 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.369119    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/636eedb83704a28c493136e55b005566-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"636eedb83704a28c493136e55b005566\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.395148 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.395089    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:18.469897 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.469850    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/22634a54757c03b5d1c07ddcc38ba334-config\") pod \"kube-apiserver-proxy-ip-10-0-140-134.ec2.internal\" (UID: \"22634a54757c03b5d1c07ddcc38ba334\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.469897 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.469883    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/22634a54757c03b5d1c07ddcc38ba334-config\") pod \"kube-apiserver-proxy-ip-10-0-140-134.ec2.internal\" (UID: \"22634a54757c03b5d1c07ddcc38ba334\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.495976 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.495948    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:18.596480 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.596450    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:18.602625 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.602610    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.607369 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.607336    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:18.697428 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.697336    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:18.797900 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.797865    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:18.895466 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.895434    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 25 00:36:18.895950 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.895608    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 25 00:36:18.895950 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.895620    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 25 00:36:18.898573 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.898556    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:18.962471 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.962376    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-24 00:31:17 +0000 UTC" deadline="2028-01-18 03:28:19.367338361 +0000 UTC"
Jul 25 00:36:18.962471 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:18.962416    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13010h52m0.404925262s"
Jul 25 00:36:18.999650 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:18.999623    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:19.018731 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.018705    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:36:19.100495 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:19.100464    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:19.125453 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:19.125413    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22634a54757c03b5d1c07ddcc38ba334.slice/crio-2251d1dc2e2c4c5d251c1fdd7371ca06fc12f36c2cec5bd549e9735655a45c9d WatchSource:0}: Error finding container 2251d1dc2e2c4c5d251c1fdd7371ca06fc12f36c2cec5bd549e9735655a45c9d: Status 404 returned error can't find the container with id 2251d1dc2e2c4c5d251c1fdd7371ca06fc12f36c2cec5bd549e9735655a45c9d
Jul 25 00:36:19.125833 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:19.125813    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636eedb83704a28c493136e55b005566.slice/crio-33bab0697c5778620ebe2171e6d620b8f7a2ab39c88df93c6de70e5f097f4aca WatchSource:0}: Error finding container 33bab0697c5778620ebe2171e6d620b8f7a2ab39c88df93c6de70e5f097f4aca: Status 404 returned error can't find the container with id 33bab0697c5778620ebe2171e6d620b8f7a2ab39c88df93c6de70e5f097f4aca
Jul 25 00:36:19.130788 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.130775    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:36:19.168272 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.168226    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" event={"ID":"636eedb83704a28c493136e55b005566","Type":"ContainerStarted","Data":"33bab0697c5778620ebe2171e6d620b8f7a2ab39c88df93c6de70e5f097f4aca"}
Jul 25 00:36:19.169218 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.169199    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" event={"ID":"22634a54757c03b5d1c07ddcc38ba334","Type":"ContainerStarted","Data":"2251d1dc2e2c4c5d251c1fdd7371ca06fc12f36c2cec5bd549e9735655a45c9d"}
Jul 25 00:36:19.201521 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:19.201500    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:19.302040 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:19.301982    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:19.402629 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:19.402598    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 25 00:36:19.415858 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.415840    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:36:19.466822 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.466803    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:19.479125 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.479083    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 25 00:36:19.480100 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.480085    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 25 00:36:19.488406 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.488391    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 25 00:36:19.943710 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.943634    2566 apiserver.go:52] "Watching apiserver"
Jul 25 00:36:19.951625 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.951592    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 25 00:36:19.952622 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.952595    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal","openshift-multus/multus-additional-cni-plugins-b5c6r","kube-system/konnectivity-agent-h8h45","openshift-multus/multus-swm5k","openshift-multus/network-metrics-daemon-bwhm2","openshift-network-diagnostics/network-check-target-gx6rn","openshift-network-operator/iptables-alerter-rjknh","openshift-ovn-kubernetes/ovnkube-node-56m27","kube-system/global-pull-secret-syncer-8vm5n","kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5","openshift-cluster-node-tuning-operator/tuned-ggm7c","openshift-dns/node-resolver-kpkfg","openshift-image-registry/node-ca-dxmdc"]
Jul 25 00:36:19.954104 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.954082    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:19.955396 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.955373    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:19.956408 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.956386    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 25 00:36:19.956510 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.956428    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 25 00:36:19.956510 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.956472    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:19.956641 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.956621    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 25 00:36:19.956746 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.956729    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qpwxd\""
Jul 25 00:36:19.957436 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.957416    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 25 00:36:19.957517 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.957500    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 25 00:36:19.958560 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.957780    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.958560 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.957865    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vsx9v\""
Jul 25 00:36:19.958560 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.958350    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 25 00:36:19.958762 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.958572    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4jgts\""
Jul 25 00:36:19.958977 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.958958    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 25 00:36:19.960702 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.959521    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:19.960702 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:19.959585    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:19.960702 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.960403    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 25 00:36:19.960702 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.960424    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 25 00:36:19.960702 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.960432    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 25 00:36:19.960702 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.960459    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cn99w\""
Jul 25 00:36:19.960702 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.960405    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 25 00:36:19.960702 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.960676    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:19.961111 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:19.960730    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:19.962148 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.962130    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:19.962238 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:19.962181    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:19.963517 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.963500    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:19.964863 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.964847    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:19.965557 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.965541    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 25 00:36:19.965834 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.965823    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 25 00:36:19.966009 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.965990    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 25 00:36:19.966234 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.966216    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.966605 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.966585    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 25 00:36:19.966927 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.966907    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sv9bh\""
Jul 25 00:36:19.967005 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.966946    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 25 00:36:19.967005 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.966985    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 25 00:36:19.967269 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.967251    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sp8wb\""
Jul 25 00:36:19.968103 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.967709    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:19.968388 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.968357    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 25 00:36:19.968494 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.968357    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 25 00:36:19.968593 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.968567    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 25 00:36:19.969027 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.969006    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:19.969536 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.969512    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 25 00:36:19.969737 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.969718    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 25 00:36:19.969813 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.969770    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 25 00:36:19.969872 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.969729    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 25 00:36:19.969994 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.969975    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 25 00:36:19.970077 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.970020    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9lhzw\""
Jul 25 00:36:19.970269 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.970219    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vq97v\""
Jul 25 00:36:19.971317 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.971297    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 25 00:36:19.971410 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.971301    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 25 00:36:19.971737 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.971705    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 25 00:36:19.978711 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.977452    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rg2k4\""
Jul 25 00:36:19.978860 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.978838    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:19.978928 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.978882    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-socket-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:19.978928 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.978915    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-socket-dir-parent\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.979024 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.978988    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnpkx\" (UniqueName: \"kubernetes.io/projected/8d1dc5e7-7792-4763-a69e-312272d69305-kube-api-access-wnpkx\") pod \"node-ca-dxmdc\" (UID: \"8d1dc5e7-7792-4763-a69e-312272d69305\") " pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:19.979123 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979084    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2cb88dd-222f-483e-8623-ead0a2448f6e-kubelet-config\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:19.979216 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979134    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:19.979216 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979166    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:19.979216 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979197    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-run-ovn-kubernetes\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.979356 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979226    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/027dd448-88fa-465e-b7e1-90d9db5cd825-env-overrides\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.979356 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979250    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-daemon-config\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.979356 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979290    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-var-lib-openvswitch\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.979356 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979319    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-registration-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:19.979356 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-run-k8s-cni-cncf-io\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.979601 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979388    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/663a5a14-c43d-4aa9-a799-03e303968608-agent-certs\") pod \"konnectivity-agent-h8h45\" (UID: \"663a5a14-c43d-4aa9-a799-03e303968608\") " pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:19.979601 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979420    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-os-release\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:19.979601 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979456    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-cni-binary-copy\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:19.979601 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979485    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-node-log\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.979601 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979516    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-device-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:19.979601 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979544    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-cnibin\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.979601 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979567    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-var-lib-kubelet\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.979914 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979634    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-run-netns\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.979914 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979666    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-system-cni-dir\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:19.979914 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979697    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnvzf\" (UniqueName: \"kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf\") pod \"network-check-target-gx6rn\" (UID: \"4be6f7d3-a562-4f8a-828c-2711679fd811\") " pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:19.979914 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979738    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/027dd448-88fa-465e-b7e1-90d9db5cd825-ovn-node-metrics-cert\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.979914 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979794    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34cc6d1e-0b4a-420e-bd17-5b5fb36ba380-tmp-dir\") pod \"node-resolver-kpkfg\" (UID: \"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380\") " pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:19.979914 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979833    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt9j7\" (UniqueName: \"kubernetes.io/projected/f4c7b19e-0f50-493b-a39c-be861ced0055-kube-api-access-vt9j7\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:19.980196 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979934    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-systemd-units\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.980196 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979965    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/027dd448-88fa-465e-b7e1-90d9db5cd825-ovnkube-script-lib\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.980196 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.979990    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-sys-fs\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:19.980196 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980023    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-run-systemd\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.980379 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980197    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-cni-netd\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.980379 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980229    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:19.980593 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980572    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d452\" (UniqueName: \"kubernetes.io/projected/dc2269d8-bbb4-49f6-8f13-823bd39398f5-kube-api-access-9d452\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.980654 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980611    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/663a5a14-c43d-4aa9-a799-03e303968608-konnectivity-ca\") pod \"konnectivity-agent-h8h45\" (UID: \"663a5a14-c43d-4aa9-a799-03e303968608\") " pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:19.980654 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980634    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-cni-dir\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.980720 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980654    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:19.980720 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980676    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:19.980720 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980697    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl4hz\" (UniqueName: \"kubernetes.io/projected/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-kube-api-access-hl4hz\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:19.980720 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980717    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc2269d8-bbb4-49f6-8f13-823bd39398f5-cni-binary-copy\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.980833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980736    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-conf-dir\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.980833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980756    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f2a318f1-0be7-4f83-b776-96364d7a952e-iptables-alerter-script\") pod \"iptables-alerter-rjknh\" (UID: \"f2a318f1-0be7-4f83-b776-96364d7a952e\") " pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:19.980833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980777    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjlfr\" (UniqueName: \"kubernetes.io/projected/027dd448-88fa-465e-b7e1-90d9db5cd825-kube-api-access-xjlfr\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.980833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980797    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d1dc5e7-7792-4763-a69e-312272d69305-host\") pod \"node-ca-dxmdc\" (UID: \"8d1dc5e7-7792-4763-a69e-312272d69305\") " pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:19.980833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980816    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8d1dc5e7-7792-4763-a69e-312272d69305-serviceca\") pod \"node-ca-dxmdc\" (UID: \"8d1dc5e7-7792-4763-a69e-312272d69305\") " pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:19.980967 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980835    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2cb88dd-222f-483e-8623-ead0a2448f6e-dbus\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:19.980967 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thll5\" (UniqueName: \"kubernetes.io/projected/f2a318f1-0be7-4f83-b776-96364d7a952e-kube-api-access-thll5\") pod \"iptables-alerter-rjknh\" (UID: \"f2a318f1-0be7-4f83-b776-96364d7a952e\") " pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:19.980967 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980875    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-etc-openvswitch\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.980967 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980895    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-run-openvswitch\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.980967 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980916    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.980967 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980936    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/027dd448-88fa-465e-b7e1-90d9db5cd825-ovnkube-config\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.980967 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980957    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-run-ovn\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.981180 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.980978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-cni-bin\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.981180 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981046    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34cc6d1e-0b4a-420e-bd17-5b5fb36ba380-hosts-file\") pod \"node-resolver-kpkfg\" (UID: \"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380\") " pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:19.981180 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981088    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-system-cni-dir\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.981180 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981111    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-var-lib-cni-multus\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.981180 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981132    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-etc-kubernetes\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.981180 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981171    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f2a318f1-0be7-4f83-b776-96364d7a952e-host-slash\") pod \"iptables-alerter-rjknh\" (UID: \"f2a318f1-0be7-4f83-b776-96364d7a952e\") " pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:19.981447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981193    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-kubelet\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.981447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981216    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-cnibin\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:19.981447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981239    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p428\" (UniqueName: \"kubernetes.io/projected/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-kube-api-access-2p428\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:19.981447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981261    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-slash\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.981447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981301    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:19.981447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981323    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn2q8\" (UniqueName: \"kubernetes.io/projected/34cc6d1e-0b4a-420e-bd17-5b5fb36ba380-kube-api-access-zn2q8\") pod \"node-resolver-kpkfg\" (UID: \"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380\") " pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:19.981447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-run-netns\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.981447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981380    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-var-lib-cni-bin\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.981447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981411    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-run-multus-certs\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.981447 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981436    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-log-socket\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:19.981967 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981458    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-os-release\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.981967 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.981480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-hostroot\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:19.983574 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.983550    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 25 00:36:19.999665 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:19.999644    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6d78k"
Jul 25 00:36:20.010451 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.010221    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6d78k"
Jul 25 00:36:20.068198 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.068167    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 25 00:36:20.082651 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.082618    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-run-netns\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.082651 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.082661    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-system-cni-dir\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.082867 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.082687    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnvzf\" (UniqueName: \"kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf\") pod \"network-check-target-gx6rn\" (UID: \"4be6f7d3-a562-4f8a-828c-2711679fd811\") " pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:20.082867 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.082721    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/027dd448-88fa-465e-b7e1-90d9db5cd825-ovn-node-metrics-cert\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.082867 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.082739    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-run-netns\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.082867 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.082797    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-system-cni-dir\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083049    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34cc6d1e-0b4a-420e-bd17-5b5fb36ba380-tmp-dir\") pod \"node-resolver-kpkfg\" (UID: \"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380\") " pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.082745    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34cc6d1e-0b4a-420e-bd17-5b5fb36ba380-tmp-dir\") pod \"node-resolver-kpkfg\" (UID: \"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380\") " pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083139    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt9j7\" (UniqueName: \"kubernetes.io/projected/f4c7b19e-0f50-493b-a39c-be861ced0055-kube-api-access-vt9j7\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083171    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-sysctl-d\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083199    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-systemd-units\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083237    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/027dd448-88fa-465e-b7e1-90d9db5cd825-ovnkube-script-lib\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083260    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-sys-fs\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083282    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-host\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-run-systemd\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083336    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-cni-netd\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083358    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083383    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d452\" (UniqueName: \"kubernetes.io/projected/dc2269d8-bbb4-49f6-8f13-823bd39398f5-kube-api-access-9d452\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083383    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083405    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-sysctl-conf\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/663a5a14-c43d-4aa9-a799-03e303968608-konnectivity-ca\") pod \"konnectivity-agent-h8h45\" (UID: \"663a5a14-c43d-4aa9-a799-03e303968608\") " pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083434    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-sys-fs\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-cni-dir\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.085952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083491    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083497    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083508    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-systemd-units\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083552    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-run-systemd\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083555    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl4hz\" (UniqueName: \"kubernetes.io/projected/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-kube-api-access-hl4hz\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.083604    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083607    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-cni-netd\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.083675    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret podName:c2cb88dd-222f-483e-8623-ead0a2448f6e nodeName:}" failed. No retries permitted until 2026-07-25 00:36:20.583643036 +0000 UTC m=+2.972165432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret") pod "global-pull-secret-syncer-8vm5n" (UID: "c2cb88dd-222f-483e-8623-ead0a2448f6e") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc2269d8-bbb4-49f6-8f13-823bd39398f5-cni-binary-copy\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083764    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-conf-dir\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083788    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f2a318f1-0be7-4f83-b776-96364d7a952e-iptables-alerter-script\") pod \"iptables-alerter-rjknh\" (UID: \"f2a318f1-0be7-4f83-b776-96364d7a952e\") " pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083811    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjlfr\" (UniqueName: \"kubernetes.io/projected/027dd448-88fa-465e-b7e1-90d9db5cd825-kube-api-access-xjlfr\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083839    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl479\" (UniqueName: \"kubernetes.io/projected/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-kube-api-access-zl479\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d1dc5e7-7792-4763-a69e-312272d69305-host\") pod \"node-ca-dxmdc\" (UID: \"8d1dc5e7-7792-4763-a69e-312272d69305\") " pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083886    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8d1dc5e7-7792-4763-a69e-312272d69305-serviceca\") pod \"node-ca-dxmdc\" (UID: \"8d1dc5e7-7792-4763-a69e-312272d69305\") " pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083908    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2cb88dd-222f-483e-8623-ead0a2448f6e-dbus\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:20.086912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thll5\" (UniqueName: \"kubernetes.io/projected/f2a318f1-0be7-4f83-b776-96364d7a952e-kube-api-access-thll5\") pod \"iptables-alerter-rjknh\" (UID: \"f2a318f1-0be7-4f83-b776-96364d7a952e\") " pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083954    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-etc-openvswitch\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.083984    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-run-openvswitch\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084025    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-run-openvswitch\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084288    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/027dd448-88fa-465e-b7e1-90d9db5cd825-ovnkube-script-lib\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.084401    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.084458    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs podName:f4c7b19e-0f50-493b-a39c-be861ced0055 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:20.584441599 +0000 UTC m=+2.972963990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs") pod "network-metrics-daemon-bwhm2" (UID: "f4c7b19e-0f50-493b-a39c-be861ced0055") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084465    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/663a5a14-c43d-4aa9-a799-03e303968608-konnectivity-ca\") pod \"konnectivity-agent-h8h45\" (UID: \"663a5a14-c43d-4aa9-a799-03e303968608\") " pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084501    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084531    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/027dd448-88fa-465e-b7e1-90d9db5cd825-ovnkube-config\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084554    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-run-ovn\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084564    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc2269d8-bbb4-49f6-8f13-823bd39398f5-cni-binary-copy\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084576    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-cni-dir\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084580    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-cni-bin\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084618    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084658    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-run-ovn\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084661    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2cb88dd-222f-483e-8623-ead0a2448f6e-dbus\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:20.087712 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084620    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d1dc5e7-7792-4763-a69e-312272d69305-host\") pod \"node-ca-dxmdc\" (UID: \"8d1dc5e7-7792-4763-a69e-312272d69305\") " pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084619    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34cc6d1e-0b4a-420e-bd17-5b5fb36ba380-hosts-file\") pod \"node-resolver-kpkfg\" (UID: \"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380\") " pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-system-cni-dir\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084707    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-var-lib-cni-multus\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-etc-kubernetes\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084764    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-sysconfig\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084790    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-kubernetes\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084790    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-etc-openvswitch\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084812    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f2a318f1-0be7-4f83-b776-96364d7a952e-host-slash\") pod \"iptables-alerter-rjknh\" (UID: \"f2a318f1-0be7-4f83-b776-96364d7a952e\") " pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084834    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-kubelet\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084858    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-tuned\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084885    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-cnibin\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084912    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p428\" (UniqueName: \"kubernetes.io/projected/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-kube-api-access-2p428\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-slash\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084946    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.084966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn2q8\" (UniqueName: \"kubernetes.io/projected/34cc6d1e-0b4a-420e-bd17-5b5fb36ba380-kube-api-access-zn2q8\") pod \"node-resolver-kpkfg\" (UID: \"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380\") " pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085010    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f2a318f1-0be7-4f83-b776-96364d7a952e-host-slash\") pod \"iptables-alerter-rjknh\" (UID: \"f2a318f1-0be7-4f83-b776-96364d7a952e\") " pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:20.088552 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085028    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f2a318f1-0be7-4f83-b776-96364d7a952e-iptables-alerter-script\") pod \"iptables-alerter-rjknh\" (UID: \"f2a318f1-0be7-4f83-b776-96364d7a952e\") " pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085039    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8d1dc5e7-7792-4763-a69e-312272d69305-serviceca\") pod \"node-ca-dxmdc\" (UID: \"8d1dc5e7-7792-4763-a69e-312272d69305\") " pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085043    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-run-netns\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085096    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-var-lib-cni-bin\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085098    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-run-netns\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085121    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-run-multus-certs\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085146    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-log-socket\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085164    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-system-cni-dir\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085186    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-os-release\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085195    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-kubelet\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085204    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-var-lib-cni-multus\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-hostroot\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085237    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-modprobe-d\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085242    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-hostroot\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085252    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-cnibin\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085262    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-lib-modules\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085287    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-var-lib-kubelet\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085301    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-run-multus-certs\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.089309 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085315    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085337    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-log-socket\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085352    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-socket-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085382    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-socket-dir-parent\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085390    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-os-release\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085405    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-run\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085432    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-etc-kubernetes\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085437    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/027dd448-88fa-465e-b7e1-90d9db5cd825-ovnkube-config\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085449    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-var-lib-cni-bin\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085487    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34cc6d1e-0b4a-420e-bd17-5b5fb36ba380-hosts-file\") pod \"node-resolver-kpkfg\" (UID: \"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380\") " pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085494    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-conf-dir\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085497    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085539    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-slash\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085519    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnpkx\" (UniqueName: \"kubernetes.io/projected/8d1dc5e7-7792-4763-a69e-312272d69305-kube-api-access-wnpkx\") pod \"node-ca-dxmdc\" (UID: \"8d1dc5e7-7792-4763-a69e-312272d69305\") " pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085597    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-cni-bin\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085605    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-socket-dir-parent\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085641    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2cb88dd-222f-483e-8623-ead0a2448f6e-kubelet-config\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085675    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.090171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085681    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2cb88dd-222f-483e-8623-ead0a2448f6e-kubelet-config\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085701    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085728    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-run-ovn-kubernetes\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085769    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-host-run-ovn-kubernetes\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085788    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-socket-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/027dd448-88fa-465e-b7e1-90d9db5cd825-env-overrides\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-daemon-config\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085874    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-systemd\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085886    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-var-lib-openvswitch\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085920    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-registration-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085949    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-run-k8s-cni-cncf-io\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085976    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-sys\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-tmp\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086018    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/663a5a14-c43d-4aa9-a799-03e303968608-agent-certs\") pod \"konnectivity-agent-h8h45\" (UID: \"663a5a14-c43d-4aa9-a799-03e303968608\") " pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086042    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-os-release\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086083    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-cni-binary-copy\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.091028 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086106    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-node-log\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-device-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086151    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-cnibin\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086174    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-var-lib-kubelet\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086268    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-os-release\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086284    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/027dd448-88fa-465e-b7e1-90d9db5cd825-env-overrides\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086308    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-registration-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.085920    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-var-lib-openvswitch\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086347    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-run-k8s-cni-cncf-io\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086393    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/027dd448-88fa-465e-b7e1-90d9db5cd825-node-log\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086567    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-device-dir\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086602    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-cnibin\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086685    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc2269d8-bbb4-49f6-8f13-823bd39398f5-host-var-lib-kubelet\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086743    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc2269d8-bbb4-49f6-8f13-823bd39398f5-multus-daemon-config\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.086823    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-cni-binary-copy\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.087138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/027dd448-88fa-465e-b7e1-90d9db5cd825-ovn-node-metrics-cert\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.088341    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:36:20.091922 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.088364    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:36:20.092733 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.088378    2566 projected.go:196] Error preparing data for projected volume kube-api-access-rnvzf for pod openshift-network-diagnostics/network-check-target-gx6rn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:20.092733 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.088432    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf podName:4be6f7d3-a562-4f8a-828c-2711679fd811 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:20.588416836 +0000 UTC m=+2.976939239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rnvzf" (UniqueName: "kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf") pod "network-check-target-gx6rn" (UID: "4be6f7d3-a562-4f8a-828c-2711679fd811") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:20.092733 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.088864    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/663a5a14-c43d-4aa9-a799-03e303968608-agent-certs\") pod \"konnectivity-agent-h8h45\" (UID: \"663a5a14-c43d-4aa9-a799-03e303968608\") " pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:20.092733 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.089132    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.092935 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.092755    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d452\" (UniqueName: \"kubernetes.io/projected/dc2269d8-bbb4-49f6-8f13-823bd39398f5-kube-api-access-9d452\") pod \"multus-swm5k\" (UID: \"dc2269d8-bbb4-49f6-8f13-823bd39398f5\") " pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.093558 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.093528    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt9j7\" (UniqueName: \"kubernetes.io/projected/f4c7b19e-0f50-493b-a39c-be861ced0055-kube-api-access-vt9j7\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:20.095103 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.095081    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn2q8\" (UniqueName: \"kubernetes.io/projected/34cc6d1e-0b4a-420e-bd17-5b5fb36ba380-kube-api-access-zn2q8\") pod \"node-resolver-kpkfg\" (UID: \"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380\") " pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:20.095103 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.095095    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl4hz\" (UniqueName: \"kubernetes.io/projected/3747cf0e-a0ef-4ac1-ac14-dd8601a518e9-kube-api-access-hl4hz\") pod \"aws-ebs-csi-driver-node-8brd5\" (UID: \"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.095380 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.095307    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thll5\" (UniqueName: \"kubernetes.io/projected/f2a318f1-0be7-4f83-b776-96364d7a952e-kube-api-access-thll5\") pod \"iptables-alerter-rjknh\" (UID: \"f2a318f1-0be7-4f83-b776-96364d7a952e\") " pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:20.097143 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.097013    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjlfr\" (UniqueName: \"kubernetes.io/projected/027dd448-88fa-465e-b7e1-90d9db5cd825-kube-api-access-xjlfr\") pod \"ovnkube-node-56m27\" (UID: \"027dd448-88fa-465e-b7e1-90d9db5cd825\") " pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.097238 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.097225    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p428\" (UniqueName: \"kubernetes.io/projected/6e35543f-8b3c-4e8c-80fc-9e1a47246e85-kube-api-access-2p428\") pod \"multus-additional-cni-plugins-b5c6r\" (UID: \"6e35543f-8b3c-4e8c-80fc-9e1a47246e85\") " pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.097563 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.097517    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnpkx\" (UniqueName: \"kubernetes.io/projected/8d1dc5e7-7792-4763-a69e-312272d69305-kube-api-access-wnpkx\") pod \"node-ca-dxmdc\" (UID: \"8d1dc5e7-7792-4763-a69e-312272d69305\") " pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:20.187079 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187028    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-sysconfig\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187079 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187085    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-kubernetes\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187109    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-tuned\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187138    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-modprobe-d\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187159    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-lib-modules\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187166    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-sysconfig\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187240    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-kubernetes\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187272    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-var-lib-kubelet\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187294    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-lib-modules\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187277    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-modprobe-d\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187311    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-run\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187318    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-var-lib-kubelet\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187351    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-systemd\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187375    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-sys\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187377    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-run\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187396    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-tmp\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187386    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-systemd\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187427    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-sys\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-sysctl-d\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187485    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-host\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187503    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-sysctl-conf\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187550    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl479\" (UniqueName: \"kubernetes.io/projected/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-kube-api-access-zl479\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187573    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-host\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187608    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-sysctl-d\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.187781 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.187730    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-sysctl-conf\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.189510 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.189489    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-tmp\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.189860 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.189836    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-etc-tuned\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.203789 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.203735    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:36:20.207329 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.207309    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl479\" (UniqueName: \"kubernetes.io/projected/afd68ca3-2b2e-4338-88e5-79be7fcf65d5-kube-api-access-zl479\") pod \"tuned-ggm7c\" (UID: \"afd68ca3-2b2e-4338-88e5-79be7fcf65d5\") " pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.266932 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.266897    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rjknh"
Jul 25 00:36:20.281936 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.281906    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kpkfg"
Jul 25 00:36:20.288604 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.288584    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:20.296329 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.296311    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-swm5k"
Jul 25 00:36:20.302805 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.302786    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dxmdc"
Jul 25 00:36:20.311449 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.311430    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b5c6r"
Jul 25 00:36:20.324997 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.324972    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:20.331641 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.331619    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5"
Jul 25 00:36:20.336428 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.336412    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ggm7c"
Jul 25 00:36:20.347591 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.347567    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:36:20.590764 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.590673    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnvzf\" (UniqueName: \"kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf\") pod \"network-check-target-gx6rn\" (UID: \"4be6f7d3-a562-4f8a-828c-2711679fd811\") " pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:20.590764 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.590724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:20.590764 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:20.590755    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:20.591044 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.590824    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:36:20.591044 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.590845    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:36:20.591044 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.590870    2566 projected.go:196] Error preparing data for projected volume kube-api-access-rnvzf for pod openshift-network-diagnostics/network-check-target-gx6rn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:20.591044 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.590880    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:20.591044 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.590932    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf podName:4be6f7d3-a562-4f8a-828c-2711679fd811 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:21.590913253 +0000 UTC m=+3.979435656 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnvzf" (UniqueName: "kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf") pod "network-check-target-gx6rn" (UID: "4be6f7d3-a562-4f8a-828c-2711679fd811") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:20.591044 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.590954    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret podName:c2cb88dd-222f-483e-8623-ead0a2448f6e nodeName:}" failed. No retries permitted until 2026-07-25 00:36:21.590944432 +0000 UTC m=+3.979466817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret") pod "global-pull-secret-syncer-8vm5n" (UID: "c2cb88dd-222f-483e-8623-ead0a2448f6e") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:20.591044 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.590956    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:20.591044 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:20.590984    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs podName:f4c7b19e-0f50-493b-a39c-be861ced0055 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:21.590972727 +0000 UTC m=+3.979495109 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs") pod "network-metrics-daemon-bwhm2" (UID: "f4c7b19e-0f50-493b-a39c-be861ced0055") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:21.013206 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.013143    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-24 00:31:19 +0000 UTC" deadline="2028-02-08 01:25:26.659548763 +0000 UTC"
Jul 25 00:36:21.013791 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.013212    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13512h49m5.646347644s"
Jul 25 00:36:21.013928 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:21.013794    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2269d8_bbb4_49f6_8f13_823bd39398f5.slice/crio-7404f25b132c32810cf2a6abeb3d76a2546fcbab5de60d547d00ece2df6a8029 WatchSource:0}: Error finding container 7404f25b132c32810cf2a6abeb3d76a2546fcbab5de60d547d00ece2df6a8029: Status 404 returned error can't find the container with id 7404f25b132c32810cf2a6abeb3d76a2546fcbab5de60d547d00ece2df6a8029
Jul 25 00:36:21.019817 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:21.019743    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd68ca3_2b2e_4338_88e5_79be7fcf65d5.slice/crio-f1a75ec4976c963ecc6a10684a0c550051106a5fd1a964ed4783c2d2e3d7e253 WatchSource:0}: Error finding container f1a75ec4976c963ecc6a10684a0c550051106a5fd1a964ed4783c2d2e3d7e253: Status 404 returned error can't find the container with id f1a75ec4976c963ecc6a10684a0c550051106a5fd1a964ed4783c2d2e3d7e253
Jul 25 00:36:21.021097 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:21.020405    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663a5a14_c43d_4aa9_a799_03e303968608.slice/crio-9b7485a27de219148248ceed383166148fe6f5c3a1c7289aac17d725e57c9685 WatchSource:0}: Error finding container 9b7485a27de219148248ceed383166148fe6f5c3a1c7289aac17d725e57c9685: Status 404 returned error can't find the container with id 9b7485a27de219148248ceed383166148fe6f5c3a1c7289aac17d725e57c9685
Jul 25 00:36:21.022041 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:21.021945    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3747cf0e_a0ef_4ac1_ac14_dd8601a518e9.slice/crio-04851e80baaf5495933cb69ad4b970bfcb6a934a9cbd811a604471b0132a0d65 WatchSource:0}: Error finding container 04851e80baaf5495933cb69ad4b970bfcb6a934a9cbd811a604471b0132a0d65: Status 404 returned error can't find the container with id 04851e80baaf5495933cb69ad4b970bfcb6a934a9cbd811a604471b0132a0d65
Jul 25 00:36:21.166298 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.166130    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:21.166469 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.166148    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:21.166469 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:21.166373    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:21.166469 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:21.166433    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:21.174053 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.174021    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" event={"ID":"22634a54757c03b5d1c07ddcc38ba334","Type":"ContainerStarted","Data":"e7bd9b28a85228aa31512b2ed00693f822a2d874acc769f91111401a0a709969"}
Jul 25 00:36:21.175269 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.175239    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56m27" event={"ID":"027dd448-88fa-465e-b7e1-90d9db5cd825","Type":"ContainerStarted","Data":"4207a678787d01189d2a5de68bb9195a3a30e890eeaa3bd8a0b8197d44689388"}
Jul 25 00:36:21.176386 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.176362    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dxmdc" event={"ID":"8d1dc5e7-7792-4763-a69e-312272d69305","Type":"ContainerStarted","Data":"4c07b8c52c696658d7106ef83253675c254de2f9c895ebc830672035ab709c71"}
Jul 25 00:36:21.177438 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.177413    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rjknh" event={"ID":"f2a318f1-0be7-4f83-b776-96364d7a952e","Type":"ContainerStarted","Data":"c82b3e8258719a771028586caedc234907b67ef110d252ce9b333d55ae41592c"}
Jul 25 00:36:21.178428 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.178407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h8h45" event={"ID":"663a5a14-c43d-4aa9-a799-03e303968608","Type":"ContainerStarted","Data":"9b7485a27de219148248ceed383166148fe6f5c3a1c7289aac17d725e57c9685"}
Jul 25 00:36:21.179454 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.179430    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5c6r" event={"ID":"6e35543f-8b3c-4e8c-80fc-9e1a47246e85","Type":"ContainerStarted","Data":"408644c97436e3348bb29ddcb1a809cd8c270777e2e9ea8208e16d5f42278b72"}
Jul 25 00:36:21.180510 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.180491    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-swm5k" event={"ID":"dc2269d8-bbb4-49f6-8f13-823bd39398f5","Type":"ContainerStarted","Data":"7404f25b132c32810cf2a6abeb3d76a2546fcbab5de60d547d00ece2df6a8029"}
Jul 25 00:36:21.181756 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.181732    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kpkfg" event={"ID":"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380","Type":"ContainerStarted","Data":"40d0e2ebc27397ad3cffc2964442876824b65b3c0ddebdc0e3b6838457d0dd99"}
Jul 25 00:36:21.182938 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.182919    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5" event={"ID":"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9","Type":"ContainerStarted","Data":"04851e80baaf5495933cb69ad4b970bfcb6a934a9cbd811a604471b0132a0d65"}
Jul 25 00:36:21.183926 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.183907    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ggm7c" event={"ID":"afd68ca3-2b2e-4338-88e5-79be7fcf65d5","Type":"ContainerStarted","Data":"f1a75ec4976c963ecc6a10684a0c550051106a5fd1a964ed4783c2d2e3d7e253"}
Jul 25 00:36:21.190852 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.190809    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" podStartSLOduration=2.190799663 podStartE2EDuration="2.190799663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:21.190343074 +0000 UTC m=+3.578865479" watchObservedRunningTime="2026-07-25 00:36:21.190799663 +0000 UTC m=+3.579322065"
Jul 25 00:36:21.598330 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.598246    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:21.598492 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.598343    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnvzf\" (UniqueName: \"kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf\") pod \"network-check-target-gx6rn\" (UID: \"4be6f7d3-a562-4f8a-828c-2711679fd811\") " pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:21.598492 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:21.598381    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:21.598599 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:21.598499    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:21.598599 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:21.598563    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs podName:f4c7b19e-0f50-493b-a39c-be861ced0055 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:23.598543982 +0000 UTC m=+5.987066369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs") pod "network-metrics-daemon-bwhm2" (UID: "f4c7b19e-0f50-493b-a39c-be861ced0055") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:21.598967 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:21.598945    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:21.599040 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:21.599001    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret podName:c2cb88dd-222f-483e-8623-ead0a2448f6e nodeName:}" failed. No retries permitted until 2026-07-25 00:36:23.598986003 +0000 UTC m=+5.987508387 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret") pod "global-pull-secret-syncer-8vm5n" (UID: "c2cb88dd-222f-483e-8623-ead0a2448f6e") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:21.599130 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:21.599091    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:36:21.599130 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:21.599105    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:36:21.599130 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:21.599118    2566 projected.go:196] Error preparing data for projected volume kube-api-access-rnvzf for pod openshift-network-diagnostics/network-check-target-gx6rn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:21.599260 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:21.599164    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf podName:4be6f7d3-a562-4f8a-828c-2711679fd811 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:23.599151061 +0000 UTC m=+5.987673444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnvzf" (UniqueName: "kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf") pod "network-check-target-gx6rn" (UID: "4be6f7d3-a562-4f8a-828c-2711679fd811") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:22.014377 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:22.014314    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-24 00:31:19 +0000 UTC" deadline="2028-05-10 06:27:38.863136108 +0000 UTC"
Jul 25 00:36:22.014377 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:22.014372    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15725h51m16.848767495s"
Jul 25 00:36:22.169184 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:22.168680    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:22.169184 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:22.168816    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:22.209596 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:22.209543    2566 generic.go:356] "Generic (PLEG): container finished" podID="636eedb83704a28c493136e55b005566" containerID="c531df90192dd58014f9b10134b0156194e22fe6ae5a8f9b5d674677ff29240f" exitCode=0
Jul 25 00:36:22.210557 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:22.210528    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" event={"ID":"636eedb83704a28c493136e55b005566","Type":"ContainerDied","Data":"c531df90192dd58014f9b10134b0156194e22fe6ae5a8f9b5d674677ff29240f"}
Jul 25 00:36:23.166850 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:23.166338    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:23.166850 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:23.166471    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:23.166850 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:23.166338    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:23.166850 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:23.166610    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:23.227987 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:23.227584    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" event={"ID":"636eedb83704a28c493136e55b005566","Type":"ContainerStarted","Data":"9d1131b027a505bbd10200f93e2bb3166dc68a41bcfe634ba30279a13921c898"}
Jul 25 00:36:23.246779 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:23.246712    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" podStartSLOduration=4.246691581 podStartE2EDuration="4.246691581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:23.246002291 +0000 UTC m=+5.634524697" watchObservedRunningTime="2026-07-25 00:36:23.246691581 +0000 UTC m=+5.635213986"
Jul 25 00:36:23.614331 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:23.614249    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnvzf\" (UniqueName: \"kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf\") pod \"network-check-target-gx6rn\" (UID: \"4be6f7d3-a562-4f8a-828c-2711679fd811\") " pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:23.614331 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:23.614311    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:23.614540 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:23.614339    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:23.614540 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:23.614490    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:23.614640 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:23.614552    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret podName:c2cb88dd-222f-483e-8623-ead0a2448f6e nodeName:}" failed. No retries permitted until 2026-07-25 00:36:27.614529507 +0000 UTC m=+10.003051894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret") pod "global-pull-secret-syncer-8vm5n" (UID: "c2cb88dd-222f-483e-8623-ead0a2448f6e") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:23.615027 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:23.614961    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:36:23.615027 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:23.614983    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:36:23.615027 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:23.614995    2566 projected.go:196] Error preparing data for projected volume kube-api-access-rnvzf for pod openshift-network-diagnostics/network-check-target-gx6rn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:23.615311 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:23.615045    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf podName:4be6f7d3-a562-4f8a-828c-2711679fd811 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:27.615030273 +0000 UTC m=+10.003552656 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnvzf" (UniqueName: "kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf") pod "network-check-target-gx6rn" (UID: "4be6f7d3-a562-4f8a-828c-2711679fd811") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:23.615311 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:23.615122    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:23.615311 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:23.615157    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs podName:f4c7b19e-0f50-493b-a39c-be861ced0055 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:27.615146403 +0000 UTC m=+10.003668788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs") pod "network-metrics-daemon-bwhm2" (UID: "f4c7b19e-0f50-493b-a39c-be861ced0055") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:24.170916 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:24.170885    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:24.171407 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:24.171025    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:25.166092 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:25.165801    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:25.166092 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:25.165927    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:25.166334 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:25.166317    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:25.166426 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:25.166407    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:26.170097 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:26.169594    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:26.170097 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:26.169739    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:27.166134 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:27.166104    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:27.166317 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:27.166221    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:27.166653 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:27.166607    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:27.166760 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:27.166709    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:27.644722    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnvzf\" (UniqueName: \"kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf\") pod \"network-check-target-gx6rn\" (UID: \"4be6f7d3-a562-4f8a-828c-2711679fd811\") " pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:27.644782    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:27.644811    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:27.644952    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:27.645018    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret podName:c2cb88dd-222f-483e-8623-ead0a2448f6e nodeName:}" failed. No retries permitted until 2026-07-25 00:36:35.644997532 +0000 UTC m=+18.033519920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret") pod "global-pull-secret-syncer-8vm5n" (UID: "c2cb88dd-222f-483e-8623-ead0a2448f6e") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:27.645328    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:27.645346    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:27.645358    2566 projected.go:196] Error preparing data for projected volume kube-api-access-rnvzf for pod openshift-network-diagnostics/network-check-target-gx6rn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:27.645401    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf podName:4be6f7d3-a562-4f8a-828c-2711679fd811 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:35.645387108 +0000 UTC m=+18.033909491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnvzf" (UniqueName: "kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf") pod "network-check-target-gx6rn" (UID: "4be6f7d3-a562-4f8a-828c-2711679fd811") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:27.645453    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:27.645506 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:27.645481    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs podName:f4c7b19e-0f50-493b-a39c-be861ced0055 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:35.645471842 +0000 UTC m=+18.033994223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs") pod "network-metrics-daemon-bwhm2" (UID: "f4c7b19e-0f50-493b-a39c-be861ced0055") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:28.167621 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:28.167145    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:28.167621 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:28.167275    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:29.166240 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:29.166205    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:29.166698 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:29.166345    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:29.166698 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:29.166386    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:29.166698 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:29.166452    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:30.165874 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:30.165831    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:30.166044 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:30.165974    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:31.165912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:31.165881    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:31.166381 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:31.165880    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:31.166381 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:31.166017    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:31.166381 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:31.166119    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:32.166350 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:32.166314    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:32.166802 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:32.166471    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:33.166253 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:33.166211    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:33.166489 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:33.166359    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:33.166489 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:33.166412    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:33.166916 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:33.166526    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:34.166545 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:34.166509    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:34.167009 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:34.166657    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:35.166267 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:35.166224    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:35.166511 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:35.166243    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:35.166511 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:35.166339    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:35.166511 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:35.166421    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:35.705882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:35.705850    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnvzf\" (UniqueName: \"kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf\") pod \"network-check-target-gx6rn\" (UID: \"4be6f7d3-a562-4f8a-828c-2711679fd811\") " pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:35.706081 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:35.705908    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:35.706081 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:35.705937    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:35.706081 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:35.706011    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:36:35.706081 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:35.706050    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:36:35.706081 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:35.706054    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:35.706081 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:35.706077    2566 projected.go:196] Error preparing data for projected volume kube-api-access-rnvzf for pod openshift-network-diagnostics/network-check-target-gx6rn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:35.706358 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:35.706055    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:35.706358 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:35.706128    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret podName:c2cb88dd-222f-483e-8623-ead0a2448f6e nodeName:}" failed. No retries permitted until 2026-07-25 00:36:51.706108168 +0000 UTC m=+34.094630569 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret") pod "global-pull-secret-syncer-8vm5n" (UID: "c2cb88dd-222f-483e-8623-ead0a2448f6e") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:35.706358 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:35.706145    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf podName:4be6f7d3-a562-4f8a-828c-2711679fd811 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:51.706136009 +0000 UTC m=+34.094658396 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnvzf" (UniqueName: "kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf") pod "network-check-target-gx6rn" (UID: "4be6f7d3-a562-4f8a-828c-2711679fd811") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:35.706358 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:35.706161    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs podName:f4c7b19e-0f50-493b-a39c-be861ced0055 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:51.706151848 +0000 UTC m=+34.094674236 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs") pod "network-metrics-daemon-bwhm2" (UID: "f4c7b19e-0f50-493b-a39c-be861ced0055") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:36.165606 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:36.165554    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:36.165797 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:36.165719    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:37.165609 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:37.165547    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:37.165609 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:37.165590    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:37.166167 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:37.165698    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:37.166167 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:37.165823    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:38.166459 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:38.166432    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:38.166952 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:38.166541    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:39.166294 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:39.166265    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:39.166482 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:39.166387    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:39.166482 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:39.166442    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:39.166867 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:39.166562    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:40.166147 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.165946    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:40.166336 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:40.166232    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:40.263879 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.263796    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5" event={"ID":"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9","Type":"ContainerStarted","Data":"6b14b63ac298925f24ce3bcdb1c45c0c17832c32ab91bd97837983b4086e2597"}
Jul 25 00:36:40.265461 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.265429    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ggm7c" event={"ID":"afd68ca3-2b2e-4338-88e5-79be7fcf65d5","Type":"ContainerStarted","Data":"a850b5cbb660f9a03f638c09dc61a9f8a79e41fcc8896871f849e463b645ed74"}
Jul 25 00:36:40.268764 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.268737    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56m27" event={"ID":"027dd448-88fa-465e-b7e1-90d9db5cd825","Type":"ContainerStarted","Data":"8fd2bd260767b109fb3fc4f9351d5d597b6c0e610b0ea4ce46ea509d7c6c3c2f"}
Jul 25 00:36:40.268891 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.268817    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56m27" event={"ID":"027dd448-88fa-465e-b7e1-90d9db5cd825","Type":"ContainerStarted","Data":"ae28eae4d414e9f5a21f57c3b4dae5044e74d691c9e67b2c22b465863695369c"}
Jul 25 00:36:40.268891 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.268872    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56m27" event={"ID":"027dd448-88fa-465e-b7e1-90d9db5cd825","Type":"ContainerStarted","Data":"f157fe543cb0b75b76a45d1a08fbf10e4c69708bba73650f5f7c7050c110dddf"}
Jul 25 00:36:40.269012 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.268918    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56m27" event={"ID":"027dd448-88fa-465e-b7e1-90d9db5cd825","Type":"ContainerStarted","Data":"2175f557bcdd3179ff97fb80e0b7b639be01b846b3635a120950684b02def5b5"}
Jul 25 00:36:40.269012 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.268968    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56m27" event={"ID":"027dd448-88fa-465e-b7e1-90d9db5cd825","Type":"ContainerStarted","Data":"7d1026f6337a3f10d1699d1cc16ae1a6903b4cb0c45f1acee8152c50589417f3"}
Jul 25 00:36:40.269141 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.269015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56m27" event={"ID":"027dd448-88fa-465e-b7e1-90d9db5cd825","Type":"ContainerStarted","Data":"16243fe5f878b64e86e149c7ab597bacb1d61b359612f78644724ed30f6b20de"}
Jul 25 00:36:40.270353 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.270326    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dxmdc" event={"ID":"8d1dc5e7-7792-4763-a69e-312272d69305","Type":"ContainerStarted","Data":"619021ef83839a16f1032fc6d4a9bd5bed22d458df0bb530f8b37a63f94b69fa"}
Jul 25 00:36:40.271778 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.271751    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h8h45" event={"ID":"663a5a14-c43d-4aa9-a799-03e303968608","Type":"ContainerStarted","Data":"61f74dcab01f9418e1e3d3b8250ba1f9d082e3c47c8bb04718e35399eca4c465"}
Jul 25 00:36:40.273521 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.273491    2566 generic.go:356] "Generic (PLEG): container finished" podID="6e35543f-8b3c-4e8c-80fc-9e1a47246e85" containerID="61521b0c6499d0c7d1b46af31085ce2560fcad3522ad905223c01f254df883ff" exitCode=0
Jul 25 00:36:40.273648 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.273598    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5c6r" event={"ID":"6e35543f-8b3c-4e8c-80fc-9e1a47246e85","Type":"ContainerDied","Data":"61521b0c6499d0c7d1b46af31085ce2560fcad3522ad905223c01f254df883ff"}
Jul 25 00:36:40.275646 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.275619    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-swm5k" event={"ID":"dc2269d8-bbb4-49f6-8f13-823bd39398f5","Type":"ContainerStarted","Data":"bde1068e84ef776c52ad009bd1cad93dbb7e9457092a5872c2e4c1985f2a954a"}
Jul 25 00:36:40.277011 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.276990    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kpkfg" event={"ID":"34cc6d1e-0b4a-420e-bd17-5b5fb36ba380","Type":"ContainerStarted","Data":"02cfdf42f52254674b24683461c3bb4f90da3a54628324b6113d78cf63587228"}
Jul 25 00:36:40.284350 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.284313    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ggm7c" podStartSLOduration=3.938040313 podStartE2EDuration="22.284301838s" totalImagesPullingTime="18.346261525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:40.284232122 +0000 UTC m=+22.672754526" watchObservedRunningTime="2026-07-25 00:36:40.284301838 +0000 UTC m=+22.672824240"
Jul 25 00:36:40.298318 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.298272    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kpkfg" podStartSLOduration=3.959972661 podStartE2EDuration="22.298258358s" totalImagesPullingTime="18.338285697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:40.297697581 +0000 UTC m=+22.686219984" watchObservedRunningTime="2026-07-25 00:36:40.298258358 +0000 UTC m=+22.686780761"
Jul 25 00:36:40.311851 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.311800    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h8h45" podStartSLOduration=11.800370455 podStartE2EDuration="22.31178092s" totalImagesPullingTime="10.511410465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:40.311756166 +0000 UTC m=+22.700278571" watchObservedRunningTime="2026-07-25 00:36:40.31178092 +0000 UTC m=+22.700303314"
Jul 25 00:36:40.331111 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.330461    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-swm5k" podStartSLOduration=3.91357108 podStartE2EDuration="22.330394026s" totalImagesPullingTime="18.416822946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:40.330108789 +0000 UTC m=+22.718631213" watchObservedRunningTime="2026-07-25 00:36:40.330394026 +0000 UTC m=+22.718916441"
Jul 25 00:36:40.344122 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.344081    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dxmdc" podStartSLOduration=4.006703742 podStartE2EDuration="22.344051438s" totalImagesPullingTime="18.337347696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:40.343966278 +0000 UTC m=+22.732488680" watchObservedRunningTime="2026-07-25 00:36:40.344051438 +0000 UTC m=+22.732573840"
Jul 25 00:36:40.774623 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:40.774595    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 25 00:36:41.048833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:41.048723    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-25T00:36:40.774619261Z","UUID":"5e06bf0d-c660-4728-8442-b222959003b6","Handler":null,"Name":"","Endpoint":""}
Jul 25 00:36:41.050189 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:41.050172    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 25 00:36:41.050324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:41.050210    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 25 00:36:41.166399 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:41.166375    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:41.166399 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:41.166396    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:41.166606 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:41.166469    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:41.166663 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:41.166609    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:41.281171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:41.281142    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rjknh" event={"ID":"f2a318f1-0be7-4f83-b776-96364d7a952e","Type":"ContainerStarted","Data":"f6c4ce107a4bdd46fb24310dece728ddd06112beff00772f3867b1d9ca48fc93"}
Jul 25 00:36:41.283400 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:41.283368    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5" event={"ID":"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9","Type":"ContainerStarted","Data":"13d1de2bba2aa7091b3a959d716572c655e840573ea2c63c649a6f4a8b2cf07e"}
Jul 25 00:36:41.295623 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:41.295580    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rjknh" podStartSLOduration=4.961262008 podStartE2EDuration="23.29556698s" totalImagesPullingTime="18.334304972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:41.295086383 +0000 UTC m=+23.683608805" watchObservedRunningTime="2026-07-25 00:36:41.29556698 +0000 UTC m=+23.684089382"
Jul 25 00:36:42.004253 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:42.004220    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:42.004763 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:42.004738    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:42.165919 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:42.165891    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:42.166106 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:42.166005    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:42.288318 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:42.288283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5" event={"ID":"3747cf0e-a0ef-4ac1-ac14-dd8601a518e9","Type":"ContainerStarted","Data":"8060ffb79c7c826d12fd74c4fb6f3cc8886432bd060cb06d76a0d15a31d390d6"}
Jul 25 00:36:42.292347 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:42.292313    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56m27" event={"ID":"027dd448-88fa-465e-b7e1-90d9db5cd825","Type":"ContainerStarted","Data":"d7b668bae41dd65f79091c56f22a5a393e90ff81afd86c18d23933d44168fcbf"}
Jul 25 00:36:42.292589 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:42.292550    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:42.293046 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:42.293028    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h8h45"
Jul 25 00:36:42.306325 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:42.306278    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8brd5" podStartSLOduration=3.802463338 podStartE2EDuration="24.306266095s" totalImagesPullingTime="20.503802757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:42.305698388 +0000 UTC m=+24.694220796" watchObservedRunningTime="2026-07-25 00:36:42.306266095 +0000 UTC m=+24.694788493"
Jul 25 00:36:43.166608 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:43.166421    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:43.166793 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:43.166426    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:43.166793 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:43.166716    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:43.166875 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:43.166788    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:44.166509 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:44.166476    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:44.166935 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:44.166618    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:45.165775 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:45.165708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:45.165775 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:45.165735    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:45.165947 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:45.165822    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:45.165947 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:45.165928    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:45.302658 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:45.302619    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56m27" event={"ID":"027dd448-88fa-465e-b7e1-90d9db5cd825","Type":"ContainerStarted","Data":"2317678804d7b83c55f81c5c68227d907a3d223d784c8defc095d78575fc74f0"}
Jul 25 00:36:45.307597 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:45.307422    2566 generic.go:356] "Generic (PLEG): container finished" podID="6e35543f-8b3c-4e8c-80fc-9e1a47246e85" containerID="a8381c7ff66d571b48762fa6873bc60035f2601703f968e3c66383b14b4dcf4e" exitCode=0
Jul 25 00:36:45.307597 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:45.307508    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5c6r" event={"ID":"6e35543f-8b3c-4e8c-80fc-9e1a47246e85","Type":"ContainerDied","Data":"a8381c7ff66d571b48762fa6873bc60035f2601703f968e3c66383b14b4dcf4e"}
Jul 25 00:36:45.329491 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:45.329456    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-56m27" podStartSLOduration=8.93020862 podStartE2EDuration="27.329445357s" totalImagesPullingTime="18.399236737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:45.329135974 +0000 UTC m=+27.717658377" watchObservedRunningTime="2026-07-25 00:36:45.329445357 +0000 UTC m=+27.717967759"
Jul 25 00:36:46.165905 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.165876    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:46.166115 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:46.165981    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:46.310522 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.310425    2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 25 00:36:46.310891 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.310839    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:46.310942 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.310918    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:46.326215 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.326191    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:46.326365 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.326301    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:46.720583 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.720533    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8vm5n"]
Jul 25 00:36:46.720774 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.720683    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:46.720839 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:46.720791    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:46.723374 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.723352    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bwhm2"]
Jul 25 00:36:46.723495 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.723438    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:46.723551 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:46.723528    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:46.723945 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.723926    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gx6rn"]
Jul 25 00:36:46.724039 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:46.724015    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:46.724144 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:46.724125    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:47.173164 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:47.173138    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:36:47.314229 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:47.314198    2566 generic.go:356] "Generic (PLEG): container finished" podID="6e35543f-8b3c-4e8c-80fc-9e1a47246e85" containerID="aaafa643eff4d46609fa041be8adf286dde425eb5f0cee6cb42e6fa6870e10df" exitCode=0
Jul 25 00:36:47.314710 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:47.314273    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5c6r" event={"ID":"6e35543f-8b3c-4e8c-80fc-9e1a47246e85","Type":"ContainerDied","Data":"aaafa643eff4d46609fa041be8adf286dde425eb5f0cee6cb42e6fa6870e10df"}
Jul 25 00:36:48.166692 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:48.166658    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:48.166885 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:48.166724    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:48.166885 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:48.166741    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:48.166885 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:48.166816    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:48.166885 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:48.166869    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:48.167092 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:48.166936    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:49.320557 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:49.320281    2566 generic.go:356] "Generic (PLEG): container finished" podID="6e35543f-8b3c-4e8c-80fc-9e1a47246e85" containerID="00b1c0e47edcb2b82f35530b1b88bf21f98e73887722c389b6a4eb3216bc8866" exitCode=0
Jul 25 00:36:49.321052 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:49.320361    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5c6r" event={"ID":"6e35543f-8b3c-4e8c-80fc-9e1a47246e85","Type":"ContainerDied","Data":"00b1c0e47edcb2b82f35530b1b88bf21f98e73887722c389b6a4eb3216bc8866"}
Jul 25 00:36:50.166434 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:50.166396    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:50.166599 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:50.166400    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:50.166599 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:50.166527    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:50.166599 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:50.166404    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:50.166790 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:50.166630    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:50.166790 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:50.166681    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:51.721520 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:51.721476    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:51.721991 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:51.721529    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:51.721991 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:51.721615    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnvzf\" (UniqueName: \"kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf\") pod \"network-check-target-gx6rn\" (UID: \"4be6f7d3-a562-4f8a-828c-2711679fd811\") " pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:51.721991 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:51.721687    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:51.721991 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:51.721725    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:36:51.721991 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:51.721729    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:51.721991 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:51.721743    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:36:51.721991 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:51.721756    2566 projected.go:196] Error preparing data for projected volume kube-api-access-rnvzf for pod openshift-network-diagnostics/network-check-target-gx6rn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:51.721991 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:51.721765    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs podName:f4c7b19e-0f50-493b-a39c-be861ced0055 nodeName:}" failed. No retries permitted until 2026-07-25 00:37:23.721742891 +0000 UTC m=+66.110265292 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs") pod "network-metrics-daemon-bwhm2" (UID: "f4c7b19e-0f50-493b-a39c-be861ced0055") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:51.721991 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:51.721785    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret podName:c2cb88dd-222f-483e-8623-ead0a2448f6e nodeName:}" failed. No retries permitted until 2026-07-25 00:37:23.721775974 +0000 UTC m=+66.110298356 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret") pod "global-pull-secret-syncer-8vm5n" (UID: "c2cb88dd-222f-483e-8623-ead0a2448f6e") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:51.721991 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:51.721801    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf podName:4be6f7d3-a562-4f8a-828c-2711679fd811 nodeName:}" failed. No retries permitted until 2026-07-25 00:37:23.721792986 +0000 UTC m=+66.110315372 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnvzf" (UniqueName: "kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf") pod "network-check-target-gx6rn" (UID: "4be6f7d3-a562-4f8a-828c-2711679fd811") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:52.166320 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:52.166278    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:52.166320 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:52.166320    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:52.166545 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:52.166400    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gx6rn" podUID="4be6f7d3-a562-4f8a-828c-2711679fd811"
Jul 25 00:36:52.166545 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:52.166441    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:52.166649 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:52.166546    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vm5n" podUID="c2cb88dd-222f-483e-8623-ead0a2448f6e"
Jul 25 00:36:52.166649 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:52.166624    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:36:52.960481 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:52.960452    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeReady"
Jul 25 00:36:52.961018 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:52.960611    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 25 00:36:53.010915 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.010890    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hw84r"]
Jul 25 00:36:53.012863 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.012844    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b9hmx"]
Jul 25 00:36:53.012979 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.012935    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.014632 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.014610    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b9hmx"
Jul 25 00:36:53.015272 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.015253    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 25 00:36:53.015676 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.015658    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b2m9l\""
Jul 25 00:36:53.015961 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.015943    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 25 00:36:53.017177 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.017158    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 25 00:36:53.017366 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.017338    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vl845\""
Jul 25 00:36:53.017455 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.017365    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 25 00:36:53.017670 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.017654    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 25 00:36:53.024767 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.024748    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b9hmx"]
Jul 25 00:36:53.028195 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.028165    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hw84r"]
Jul 25 00:36:53.133616 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.133578    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxxp7\" (UniqueName: \"kubernetes.io/projected/7de09553-7361-41fb-b3e3-2776190c44c7-kube-api-access-gxxp7\") pod \"ingress-canary-b9hmx\" (UID: \"7de09553-7361-41fb-b3e3-2776190c44c7\") " pod="openshift-ingress-canary/ingress-canary-b9hmx"
Jul 25 00:36:53.133833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.133633    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7839bc1-c826-40cd-9428-4e7080474366-tmp-dir\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.133833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.133681    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7de09553-7361-41fb-b3e3-2776190c44c7-cert\") pod \"ingress-canary-b9hmx\" (UID: \"7de09553-7361-41fb-b3e3-2776190c44c7\") " pod="openshift-ingress-canary/ingress-canary-b9hmx"
Jul 25 00:36:53.133833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.133700    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnktq\" (UniqueName: \"kubernetes.io/projected/e7839bc1-c826-40cd-9428-4e7080474366-kube-api-access-xnktq\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.133833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.133719    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7839bc1-c826-40cd-9428-4e7080474366-config-volume\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.133833 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.133749    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.234777 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.234755    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxxp7\" (UniqueName: \"kubernetes.io/projected/7de09553-7361-41fb-b3e3-2776190c44c7-kube-api-access-gxxp7\") pod \"ingress-canary-b9hmx\" (UID: \"7de09553-7361-41fb-b3e3-2776190c44c7\") " pod="openshift-ingress-canary/ingress-canary-b9hmx"
Jul 25 00:36:53.234859 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.234796    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7839bc1-c826-40cd-9428-4e7080474366-tmp-dir\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.234859 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.234832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7de09553-7361-41fb-b3e3-2776190c44c7-cert\") pod \"ingress-canary-b9hmx\" (UID: \"7de09553-7361-41fb-b3e3-2776190c44c7\") " pod="openshift-ingress-canary/ingress-canary-b9hmx"
Jul 25 00:36:53.234859 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.234848    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnktq\" (UniqueName: \"kubernetes.io/projected/e7839bc1-c826-40cd-9428-4e7080474366-kube-api-access-xnktq\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.234973 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.234864    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7839bc1-c826-40cd-9428-4e7080474366-config-volume\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.234973 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.234916    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.235043 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:53.235016    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:53.235113 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:53.235080    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls podName:e7839bc1-c826-40cd-9428-4e7080474366 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:53.735048502 +0000 UTC m=+36.123570890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls") pod "dns-default-hw84r" (UID: "e7839bc1-c826-40cd-9428-4e7080474366") : secret "dns-default-metrics-tls" not found
Jul 25 00:36:53.235310 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.235287    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7839bc1-c826-40cd-9428-4e7080474366-tmp-dir\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.235492 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.235475    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7839bc1-c826-40cd-9428-4e7080474366-config-volume\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.238869 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.238848    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7de09553-7361-41fb-b3e3-2776190c44c7-cert\") pod \"ingress-canary-b9hmx\" (UID: \"7de09553-7361-41fb-b3e3-2776190c44c7\") " pod="openshift-ingress-canary/ingress-canary-b9hmx"
Jul 25 00:36:53.244477 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.244454    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnktq\" (UniqueName: \"kubernetes.io/projected/e7839bc1-c826-40cd-9428-4e7080474366-kube-api-access-xnktq\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.244642 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.244624    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxxp7\" (UniqueName: \"kubernetes.io/projected/7de09553-7361-41fb-b3e3-2776190c44c7-kube-api-access-gxxp7\") pod \"ingress-canary-b9hmx\" (UID: \"7de09553-7361-41fb-b3e3-2776190c44c7\") " pod="openshift-ingress-canary/ingress-canary-b9hmx"
Jul 25 00:36:53.332244 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.332221    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b9hmx"
Jul 25 00:36:53.336453 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.336429    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5c6r" event={"ID":"6e35543f-8b3c-4e8c-80fc-9e1a47246e85","Type":"ContainerStarted","Data":"ac0be1b7bb9d07f88d832da1beaaf8b447fff58d350734adc044e9f19c8bb64c"}
Jul 25 00:36:53.458051 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.457866    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b9hmx"]
Jul 25 00:36:53.462798 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:36:53.462761    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de09553_7361_41fb_b3e3_2776190c44c7.slice/crio-474b425581827c8129612825a4bee4ecace351e10f16f98c59831f0a81663c7a WatchSource:0}: Error finding container 474b425581827c8129612825a4bee4ecace351e10f16f98c59831f0a81663c7a: Status 404 returned error can't find the container with id 474b425581827c8129612825a4bee4ecace351e10f16f98c59831f0a81663c7a
Jul 25 00:36:53.739383 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:53.739290    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:53.739541 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:53.739425    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:53.739541 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:53.739484    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls podName:e7839bc1-c826-40cd-9428-4e7080474366 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:54.739468746 +0000 UTC m=+37.127991130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls") pod "dns-default-hw84r" (UID: "e7839bc1-c826-40cd-9428-4e7080474366") : secret "dns-default-metrics-tls" not found
Jul 25 00:36:54.165797 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.165760    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:36:54.166439 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.165760    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:36:54.166439 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.165760    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:36:54.168830 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.168810    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 25 00:36:54.168958 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.168810    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 25 00:36:54.169714 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.169693    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7xdtb\""
Jul 25 00:36:54.169990 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.169974    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 25 00:36:54.170096 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.170016    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9kt6t\""
Jul 25 00:36:54.170159 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.170091    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 25 00:36:54.341839 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.341805    2566 generic.go:356] "Generic (PLEG): container finished" podID="6e35543f-8b3c-4e8c-80fc-9e1a47246e85" containerID="ac0be1b7bb9d07f88d832da1beaaf8b447fff58d350734adc044e9f19c8bb64c" exitCode=0
Jul 25 00:36:54.342019 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.341894    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5c6r" event={"ID":"6e35543f-8b3c-4e8c-80fc-9e1a47246e85","Type":"ContainerDied","Data":"ac0be1b7bb9d07f88d832da1beaaf8b447fff58d350734adc044e9f19c8bb64c"}
Jul 25 00:36:54.343278 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.343192    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b9hmx" event={"ID":"7de09553-7361-41fb-b3e3-2776190c44c7","Type":"ContainerStarted","Data":"474b425581827c8129612825a4bee4ecace351e10f16f98c59831f0a81663c7a"}
Jul 25 00:36:54.748054 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:54.747973    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:54.748210 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:54.748120    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:54.748263 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:54.748208    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls podName:e7839bc1-c826-40cd-9428-4e7080474366 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:56.74818688 +0000 UTC m=+39.136709262 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls") pod "dns-default-hw84r" (UID: "e7839bc1-c826-40cd-9428-4e7080474366") : secret "dns-default-metrics-tls" not found
Jul 25 00:36:55.348806 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:55.348773    2566 generic.go:356] "Generic (PLEG): container finished" podID="6e35543f-8b3c-4e8c-80fc-9e1a47246e85" containerID="51652b5daf33c3457a5661d99af4a6cc189c33129ee7956097e2bffd4763d628" exitCode=0
Jul 25 00:36:55.349300 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:55.348827    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5c6r" event={"ID":"6e35543f-8b3c-4e8c-80fc-9e1a47246e85","Type":"ContainerDied","Data":"51652b5daf33c3457a5661d99af4a6cc189c33129ee7956097e2bffd4763d628"}
Jul 25 00:36:56.355029 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:56.354743    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b5c6r" event={"ID":"6e35543f-8b3c-4e8c-80fc-9e1a47246e85","Type":"ContainerStarted","Data":"a1548e07f8c0867e2d38df5e935e0bb7539a1e93f139d720c4606bd7b0b354fa"}
Jul 25 00:36:56.356274 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:56.356248    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b9hmx" event={"ID":"7de09553-7361-41fb-b3e3-2776190c44c7","Type":"ContainerStarted","Data":"d4bf60373e9fca48d722d6c4e6613e36cda5125bed9feeee53678e43f16fc651"}
Jul 25 00:36:56.377599 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:56.377532    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b5c6r" podStartSLOduration=9.400095082 podStartE2EDuration="38.377512731s" totalImagesPullingTime="28.977417649s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:56.376226918 +0000 UTC m=+38.764749319" watchObservedRunningTime="2026-07-25 00:36:56.377512731 +0000 UTC m=+38.766035124"
Jul 25 00:36:56.394008 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:56.393963    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b9hmx" podStartSLOduration=1.7421875660000001 podStartE2EDuration="4.393946121s" totalImagesPullingTime="2.651758555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:56.393591389 +0000 UTC m=+38.782113793" watchObservedRunningTime="2026-07-25 00:36:56.393946121 +0000 UTC m=+38.782468525"
Jul 25 00:36:56.764228 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:36:56.764140    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:36:56.764382 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:56.764262    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:56.764382 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:36:56.764317    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls podName:e7839bc1-c826-40cd-9428-4e7080474366 nodeName:}" failed. No retries permitted until 2026-07-25 00:37:00.764303131 +0000 UTC m=+43.152825516 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls") pod "dns-default-hw84r" (UID: "e7839bc1-c826-40cd-9428-4e7080474366") : secret "dns-default-metrics-tls" not found
Jul 25 00:37:00.793342 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:00.793304    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:37:00.793752 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:00.793464    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:37:00.793752 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:00.793529    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls podName:e7839bc1-c826-40cd-9428-4e7080474366 nodeName:}" failed. No retries permitted until 2026-07-25 00:37:08.793514439 +0000 UTC m=+51.182036821 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls") pod "dns-default-hw84r" (UID: "e7839bc1-c826-40cd-9428-4e7080474366") : secret "dns-default-metrics-tls" not found
Jul 25 00:37:08.847860 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:08.847822    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:37:08.848347 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:08.847969    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:37:08.848347 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:08.848031    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls podName:e7839bc1-c826-40cd-9428-4e7080474366 nodeName:}" failed. No retries permitted until 2026-07-25 00:37:24.848015767 +0000 UTC m=+67.236538148 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls") pod "dns-default-hw84r" (UID: "e7839bc1-c826-40cd-9428-4e7080474366") : secret "dns-default-metrics-tls" not found
Jul 25 00:37:18.336135 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:18.336103    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-56m27"
Jul 25 00:37:21.729202 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.729171    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"]
Jul 25 00:37:21.731719 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.731694    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:21.731719 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.731716    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"]
Jul 25 00:37:21.733601 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.733496    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.737105 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:21.737075    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"open-cluster-management-image-pull-credentials\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret"
Jul 25 00:37:21.738436 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.738418    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 25 00:37:21.739314 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.739296    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 25 00:37:21.740999 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.740981    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 25 00:37:21.742181 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.742165    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 25 00:37:21.745086 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.745054    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 25 00:37:21.745279 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.745265    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 25 00:37:21.749388 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.749374    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 25 00:37:21.777020 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.777000    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"]
Jul 25 00:37:21.786789 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.786772    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"]
Jul 25 00:37:21.836150 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.836125    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-hub\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.836249 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.836171    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4q6d\" (UniqueName: \"kubernetes.io/projected/7fd8a802-c8db-41f0-9c76-98fc310451dc-kube-api-access-m4q6d\") pod \"klusterlet-addon-workmgr-78b4dffd9b-js2qg\" (UID: \"7fd8a802-c8db-41f0-9c76-98fc310451dc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:21.836249 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.836203    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/063b0345-4b93-4e03-8c72-2bca6c04f2ab-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.836249 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.836221    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fd8a802-c8db-41f0-9c76-98fc310451dc-tmp\") pod \"klusterlet-addon-workmgr-78b4dffd9b-js2qg\" (UID: \"7fd8a802-c8db-41f0-9c76-98fc310451dc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:21.836249 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.836240    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.836444 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.836266    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7fd8a802-c8db-41f0-9c76-98fc310451dc-klusterlet-config\") pod \"klusterlet-addon-workmgr-78b4dffd9b-js2qg\" (UID: \"7fd8a802-c8db-41f0-9c76-98fc310451dc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:21.836444 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.836338    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-ca\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.836444 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.836372    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q782\" (UniqueName: \"kubernetes.io/projected/063b0345-4b93-4e03-8c72-2bca6c04f2ab-kube-api-access-5q782\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.836444 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.836415    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.937126 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-ca\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.937209 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937130    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q782\" (UniqueName: \"kubernetes.io/projected/063b0345-4b93-4e03-8c72-2bca6c04f2ab-kube-api-access-5q782\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.937209 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937159    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.937209 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937183    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-hub\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.937318 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937230    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4q6d\" (UniqueName: \"kubernetes.io/projected/7fd8a802-c8db-41f0-9c76-98fc310451dc-kube-api-access-m4q6d\") pod \"klusterlet-addon-workmgr-78b4dffd9b-js2qg\" (UID: \"7fd8a802-c8db-41f0-9c76-98fc310451dc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:21.937318 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937269    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/063b0345-4b93-4e03-8c72-2bca6c04f2ab-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.937678 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937294    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fd8a802-c8db-41f0-9c76-98fc310451dc-tmp\") pod \"klusterlet-addon-workmgr-78b4dffd9b-js2qg\" (UID: \"7fd8a802-c8db-41f0-9c76-98fc310451dc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:21.937767 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937678    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fd8a802-c8db-41f0-9c76-98fc310451dc-tmp\") pod \"klusterlet-addon-workmgr-78b4dffd9b-js2qg\" (UID: \"7fd8a802-c8db-41f0-9c76-98fc310451dc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:21.937767 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937714    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.937852 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937767    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7fd8a802-c8db-41f0-9c76-98fc310451dc-klusterlet-config\") pod \"klusterlet-addon-workmgr-78b4dffd9b-js2qg\" (UID: \"7fd8a802-c8db-41f0-9c76-98fc310451dc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:21.938000 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.937980    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/063b0345-4b93-4e03-8c72-2bca6c04f2ab-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.941151 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.941126    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.941254 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.941235    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-ca\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.941254 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.941246    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-hub\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.941333 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.941304    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7fd8a802-c8db-41f0-9c76-98fc310451dc-klusterlet-config\") pod \"klusterlet-addon-workmgr-78b4dffd9b-js2qg\" (UID: \"7fd8a802-c8db-41f0-9c76-98fc310451dc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:21.941367 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.941342    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/063b0345-4b93-4e03-8c72-2bca6c04f2ab-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.944492 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.944473    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q782\" (UniqueName: \"kubernetes.io/projected/063b0345-4b93-4e03-8c72-2bca6c04f2ab-kube-api-access-5q782\") pod \"cluster-proxy-proxy-agent-cbd567f95-cn4lm\" (UID: \"063b0345-4b93-4e03-8c72-2bca6c04f2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:21.944813 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:21.944793    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4q6d\" (UniqueName: \"kubernetes.io/projected/7fd8a802-c8db-41f0-9c76-98fc310451dc-kube-api-access-m4q6d\") pod \"klusterlet-addon-workmgr-78b4dffd9b-js2qg\" (UID: \"7fd8a802-c8db-41f0-9c76-98fc310451dc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:23.042310 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.042271    2566 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Jul 25 00:37:23.042673 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.042348    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:23.058891 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.058865    2566 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Jul 25 00:37:23.059008 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.058915    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:37:23.171194 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.171094    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"]
Jul 25 00:37:23.173698 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:37:23.173667    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fd8a802_c8db_41f0_9c76_98fc310451dc.slice/crio-7b0a324a0d80097c06d5db271056d2b4735e8b2ead87300c6a8016a327344033 WatchSource:0}: Error finding container 7b0a324a0d80097c06d5db271056d2b4735e8b2ead87300c6a8016a327344033: Status 404 returned error can't find the container with id 7b0a324a0d80097c06d5db271056d2b4735e8b2ead87300c6a8016a327344033
Jul 25 00:37:23.186818 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.186796    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"]
Jul 25 00:37:23.189923 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:37:23.189895    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod063b0345_4b93_4e03_8c72_2bca6c04f2ab.slice/crio-be372d0d63cba2c390b93bc880ce301139b9ea736b596bdba4f338af6300396b WatchSource:0}: Error finding container be372d0d63cba2c390b93bc880ce301139b9ea736b596bdba4f338af6300396b: Status 404 returned error can't find the container with id be372d0d63cba2c390b93bc880ce301139b9ea736b596bdba4f338af6300396b
Jul 25 00:37:23.251499 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.251479    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 25 00:37:23.417443 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.417418    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" event={"ID":"063b0345-4b93-4e03-8c72-2bca6c04f2ab","Type":"ContainerStarted","Data":"be372d0d63cba2c390b93bc880ce301139b9ea736b596bdba4f338af6300396b"}
Jul 25 00:37:23.418477 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.418458    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg" event={"ID":"7fd8a802-c8db-41f0-9c76-98fc310451dc","Type":"ContainerStarted","Data":"7b0a324a0d80097c06d5db271056d2b4735e8b2ead87300c6a8016a327344033"}
Jul 25 00:37:23.752623 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.752568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:37:23.752623 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.752599    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:37:23.752793 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.752642    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnvzf\" (UniqueName: \"kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf\") pod \"network-check-target-gx6rn\" (UID: \"4be6f7d3-a562-4f8a-828c-2711679fd811\") " pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:37:23.755040 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.755022    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 25 00:37:23.755136 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.755024    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 25 00:37:23.755136 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.755030    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 25 00:37:23.762820 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:23.762796    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 25 00:37:23.762907 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:23.762846    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs podName:f4c7b19e-0f50-493b-a39c-be861ced0055 nodeName:}" failed. No retries permitted until 2026-07-25 00:38:27.762832114 +0000 UTC m=+130.151354495 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs") pod "network-metrics-daemon-bwhm2" (UID: "f4c7b19e-0f50-493b-a39c-be861ced0055") : secret "metrics-daemon-secret" not found
Jul 25 00:37:23.764589 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.764570    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 25 00:37:23.764965 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.764941    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2cb88dd-222f-483e-8623-ead0a2448f6e-original-pull-secret\") pod \"global-pull-secret-syncer-8vm5n\" (UID: \"c2cb88dd-222f-483e-8623-ead0a2448f6e\") " pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:37:23.775120 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.775102    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnvzf\" (UniqueName: \"kubernetes.io/projected/4be6f7d3-a562-4f8a-828c-2711679fd811-kube-api-access-rnvzf\") pod \"network-check-target-gx6rn\" (UID: \"4be6f7d3-a562-4f8a-828c-2711679fd811\") " pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:37:23.887426 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.887408    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9kt6t\""
Jul 25 00:37:23.896317 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.896295    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:37:23.898330 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:23.898310    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vm5n"
Jul 25 00:37:24.021847 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:24.021747    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8vm5n"]
Jul 25 00:37:24.041981 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:24.041856    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gx6rn"]
Jul 25 00:37:24.044808 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:37:24.044780    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4be6f7d3_a562_4f8a_828c_2711679fd811.slice/crio-f830188ad44eeae5b0aae53f5b9ad28f1cd0a83040d428f51aba672243559b76 WatchSource:0}: Error finding container f830188ad44eeae5b0aae53f5b9ad28f1cd0a83040d428f51aba672243559b76: Status 404 returned error can't find the container with id f830188ad44eeae5b0aae53f5b9ad28f1cd0a83040d428f51aba672243559b76
Jul 25 00:37:24.423324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:24.423287    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8vm5n" event={"ID":"c2cb88dd-222f-483e-8623-ead0a2448f6e","Type":"ContainerStarted","Data":"52526400a1589b0e8cf6681c255d5acf8f980340e1b805425ffe5343982705fb"}
Jul 25 00:37:24.425297 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:24.425264    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gx6rn" event={"ID":"4be6f7d3-a562-4f8a-828c-2711679fd811","Type":"ContainerStarted","Data":"f830188ad44eeae5b0aae53f5b9ad28f1cd0a83040d428f51aba672243559b76"}
Jul 25 00:37:24.863089 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:24.862989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:37:24.863242 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:24.863192    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:37:24.863297 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:24.863285    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls podName:e7839bc1-c826-40cd-9428-4e7080474366 nodeName:}" failed. No retries permitted until 2026-07-25 00:37:56.863262563 +0000 UTC m=+99.251784964 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls") pod "dns-default-hw84r" (UID: "e7839bc1-c826-40cd-9428-4e7080474366") : secret "dns-default-metrics-tls" not found
Jul 25 00:37:32.450130 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.450090    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg" event={"ID":"7fd8a802-c8db-41f0-9c76-98fc310451dc","Type":"ContainerStarted","Data":"05a663b971857573452e56f37b645e77c182b45932bbdaadf4fe156f162d7e04"}
Jul 25 00:37:32.450585 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.450282    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:32.451727 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.451701    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gx6rn" event={"ID":"4be6f7d3-a562-4f8a-828c-2711679fd811","Type":"ContainerStarted","Data":"a8aed0bca8295b770e15eeaf10968fbee3182e1b390f7fe893744533d889f335"}
Jul 25 00:37:32.451849 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.451784    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:37:32.452128 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.452102    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:37:32.453762 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.453742    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" event={"ID":"063b0345-4b93-4e03-8c72-2bca6c04f2ab","Type":"ContainerStarted","Data":"eefd2c99437a0d58bb347d8ea6a9f9d151ed9382c5f4918f3b4b4457f3127812"}
Jul 25 00:37:32.453845 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.453775    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" event={"ID":"063b0345-4b93-4e03-8c72-2bca6c04f2ab","Type":"ContainerStarted","Data":"020784b794df0711278f88e64dfe5ce8a6d49e454e4bcbb6a8d1ee3b8b2c0d14"}
Jul 25 00:37:32.453845 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.453789    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" event={"ID":"063b0345-4b93-4e03-8c72-2bca6c04f2ab","Type":"ContainerStarted","Data":"575abb952db5aa952d73345d191dcbc6793f8ca2b9e2bcffcafc00aae3b6244b"}
Jul 25 00:37:32.455006 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.454989    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8vm5n" event={"ID":"c2cb88dd-222f-483e-8623-ead0a2448f6e","Type":"ContainerStarted","Data":"f11c102f1e41e440b60277aad23d2a7b9260b6c6dd8999b441d465d28000d886"}
Jul 25 00:37:32.467648 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.467609    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg" podStartSLOduration=3.054383308 podStartE2EDuration="11.467595723s" totalImagesPullingTime="8.413212415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:37:32.46619395 +0000 UTC m=+74.854716353" watchObservedRunningTime="2026-07-25 00:37:32.467595723 +0000 UTC m=+74.856118127"
Jul 25 00:37:32.501333 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.501284    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" podStartSLOduration=3.082029685 podStartE2EDuration="11.501267271s" totalImagesPullingTime="8.419237586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:37:32.499799546 +0000 UTC m=+74.888321939" watchObservedRunningTime="2026-07-25 00:37:32.501267271 +0000 UTC m=+74.889789676"
Jul 25 00:37:32.516298 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.516261    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gx6rn" podStartSLOduration=66.966545657 podStartE2EDuration="1m14.516250448s" totalImagesPullingTime="7.549704791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:37:32.515956855 +0000 UTC m=+74.904479258" watchObservedRunningTime="2026-07-25 00:37:32.516250448 +0000 UTC m=+74.904772877"
Jul 25 00:37:32.530356 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:32.530316    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8vm5n" podStartSLOduration=66.952693684 podStartE2EDuration="1m14.530305678s" totalImagesPullingTime="7.577611994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:37:32.530102367 +0000 UTC m=+74.918624771" watchObservedRunningTime="2026-07-25 00:37:32.530305678 +0000 UTC m=+74.918828081"
Jul 25 00:37:56.876558 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:37:56.876516    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:37:56.876947 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:56.876677    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:37:56.876947 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:37:56.876750    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls podName:e7839bc1-c826-40cd-9428-4e7080474366 nodeName:}" failed. No retries permitted until 2026-07-25 00:39:00.876732002 +0000 UTC m=+163.265254399 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls") pod "dns-default-hw84r" (UID: "e7839bc1-c826-40cd-9428-4e7080474366") : secret "dns-default-metrics-tls" not found
Jul 25 00:38:03.461060 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:38:03.461031    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gx6rn"
Jul 25 00:38:27.769805 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:38:27.769760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:38:27.770306 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:38:27.769881    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 25 00:38:27.770306 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:38:27.769934    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs podName:f4c7b19e-0f50-493b-a39c-be861ced0055 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:29.769920029 +0000 UTC m=+252.158442410 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs") pod "network-metrics-daemon-bwhm2" (UID: "f4c7b19e-0f50-493b-a39c-be861ced0055") : secret "metrics-daemon-secret" not found
Jul 25 00:38:56.025500 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:38:56.025464    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-hw84r" podUID="e7839bc1-c826-40cd-9428-4e7080474366"
Jul 25 00:38:56.676115 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:38:56.676092    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hw84r"
Jul 25 00:38:57.178767 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:38:57.178742    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-bwhm2" podUID="f4c7b19e-0f50-493b-a39c-be861ced0055"
Jul 25 00:39:00.880779 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:39:00.880743    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:39:00.881248 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:39:00.880882    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:39:00.881248 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:39:00.880949    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls podName:e7839bc1-c826-40cd-9428-4e7080474366 nodeName:}" failed. No retries permitted until 2026-07-25 00:41:02.880930082 +0000 UTC m=+285.269452467 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls") pod "dns-default-hw84r" (UID: "e7839bc1-c826-40cd-9428-4e7080474366") : secret "dns-default-metrics-tls" not found
Jul 25 00:39:12.165937 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:39:12.165866    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:39:31.762460 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:39:31.762431    2566 generic.go:356] "Generic (PLEG): container finished" podID="7fd8a802-c8db-41f0-9c76-98fc310451dc" containerID="05a663b971857573452e56f37b645e77c182b45932bbdaadf4fe156f162d7e04" exitCode=1
Jul 25 00:39:31.762770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:39:31.762489    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg" event={"ID":"7fd8a802-c8db-41f0-9c76-98fc310451dc","Type":"ContainerDied","Data":"05a663b971857573452e56f37b645e77c182b45932bbdaadf4fe156f162d7e04"}
Jul 25 00:39:31.762770 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:39:31.762754    2566 scope.go:117] "RemoveContainer" containerID="05a663b971857573452e56f37b645e77c182b45932bbdaadf4fe156f162d7e04"
Jul 25 00:39:32.450528 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:39:32.450500    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:39:32.766641 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:39:32.766583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg" event={"ID":"7fd8a802-c8db-41f0-9c76-98fc310451dc","Type":"ContainerStarted","Data":"020a0110c43d948f779bb0012f9c571da27fac67fa65f72f90090f47a4fc93f4"}
Jul 25 00:39:32.766997 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:39:32.766774    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:39:32.767304 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:39:32.767287    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b4dffd9b-js2qg"
Jul 25 00:40:14.238895 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.238856    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6xwds"]
Jul 25 00:40:14.240893 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.240872    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.241111 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.241050    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-crio-socket\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.241295 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.241135    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-data-volume\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.241295 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.241236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.241295 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.241278    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.241474 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.241317    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmfsj\" (UniqueName: \"kubernetes.io/projected/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-api-access-fmfsj\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.242950 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.242921    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 25 00:40:14.243100 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.243058    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6x25j\""
Jul 25 00:40:14.243199 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.243179    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 25 00:40:14.243918 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.243893    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 25 00:40:14.244030 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.243920    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 25 00:40:14.252786 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.252758    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6xwds"]
Jul 25 00:40:14.342417 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.342385    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.342537 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.342421    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.342537 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.342453    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmfsj\" (UniqueName: \"kubernetes.io/projected/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-api-access-fmfsj\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.342537 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.342483    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-crio-socket\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.342537 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.342502    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-data-volume\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.342537 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:14.342509    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 00:40:14.342869 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:14.342565    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:14.342869 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:14.342592    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm podName:fc9f4877-ba9e-46f0-8716-0564f6b1f79b nodeName:}" failed. No retries permitted until 2026-07-25 00:40:14.842573972 +0000 UTC m=+237.231096354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xwds" (UID: "fc9f4877-ba9e-46f0-8716-0564f6b1f79b") : configmap "kube-rbac-proxy" not found
Jul 25 00:40:14.342869 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:14.342632    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls podName:fc9f4877-ba9e-46f0-8716-0564f6b1f79b nodeName:}" failed. No retries permitted until 2026-07-25 00:40:14.842612408 +0000 UTC m=+237.231134854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xwds" (UID: "fc9f4877-ba9e-46f0-8716-0564f6b1f79b") : secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:14.342869 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.342629    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-crio-socket\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.342869 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.342763    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-data-volume\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.352807 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.352779    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmfsj\" (UniqueName: \"kubernetes.io/projected/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-api-access-fmfsj\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.844691 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.844671    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.844786 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:14.844698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:14.844786 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:14.844774    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 00:40:14.844854 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:14.844830    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm podName:fc9f4877-ba9e-46f0-8716-0564f6b1f79b nodeName:}" failed. No retries permitted until 2026-07-25 00:40:15.844818213 +0000 UTC m=+238.233340599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xwds" (UID: "fc9f4877-ba9e-46f0-8716-0564f6b1f79b") : configmap "kube-rbac-proxy" not found
Jul 25 00:40:14.844854 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:14.844833    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:14.844919 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:14.844866    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls podName:fc9f4877-ba9e-46f0-8716-0564f6b1f79b nodeName:}" failed. No retries permitted until 2026-07-25 00:40:15.84485641 +0000 UTC m=+238.233378791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xwds" (UID: "fc9f4877-ba9e-46f0-8716-0564f6b1f79b") : secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:15.850402 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:15.850372    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:15.850718 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:15.850441    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:15.850718 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:15.850487    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 00:40:15.850718 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:15.850496    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:15.850718 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:15.850522    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm podName:fc9f4877-ba9e-46f0-8716-0564f6b1f79b nodeName:}" failed. No retries permitted until 2026-07-25 00:40:17.850511637 +0000 UTC m=+240.239034019 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xwds" (UID: "fc9f4877-ba9e-46f0-8716-0564f6b1f79b") : configmap "kube-rbac-proxy" not found
Jul 25 00:40:15.850718 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:15.850534    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls podName:fc9f4877-ba9e-46f0-8716-0564f6b1f79b nodeName:}" failed. No retries permitted until 2026-07-25 00:40:17.850528723 +0000 UTC m=+240.239051105 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xwds" (UID: "fc9f4877-ba9e-46f0-8716-0564f6b1f79b") : secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:17.221249 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:17.221228    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kpkfg_34cc6d1e-0b4a-420e-bd17-5b5fb36ba380/dns-node-resolver/0.log"
Jul 25 00:40:17.820843 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:17.820823    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dxmdc_8d1dc5e7-7792-4763-a69e-312272d69305/node-ca/0.log"
Jul 25 00:40:17.861703 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:17.861681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:17.861803 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:17.861743    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:17.861803 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:17.861793    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 00:40:17.861876 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:17.861806    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:17.861876 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:17.861845    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm podName:fc9f4877-ba9e-46f0-8716-0564f6b1f79b nodeName:}" failed. No retries permitted until 2026-07-25 00:40:21.861833813 +0000 UTC m=+244.250356195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xwds" (UID: "fc9f4877-ba9e-46f0-8716-0564f6b1f79b") : configmap "kube-rbac-proxy" not found
Jul 25 00:40:17.861876 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:17.861857    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls podName:fc9f4877-ba9e-46f0-8716-0564f6b1f79b nodeName:}" failed. No retries permitted until 2026-07-25 00:40:21.861851212 +0000 UTC m=+244.250373593 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xwds" (UID: "fc9f4877-ba9e-46f0-8716-0564f6b1f79b") : secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:18.621568 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:18.621539    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b9hmx_7de09553-7361-41fb-b3e3-2776190c44c7/serve-healthcheck-canary/0.log"
Jul 25 00:40:21.890390 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:21.890352    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:21.890390 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:21.890389    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:21.890836 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:21.890463    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 00:40:21.890836 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:21.890488    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:21.890836 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:21.890522    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm podName:fc9f4877-ba9e-46f0-8716-0564f6b1f79b nodeName:}" failed. No retries permitted until 2026-07-25 00:40:29.890505046 +0000 UTC m=+252.279027431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xwds" (UID: "fc9f4877-ba9e-46f0-8716-0564f6b1f79b") : configmap "kube-rbac-proxy" not found
Jul 25 00:40:21.890836 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:21.890538    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls podName:fc9f4877-ba9e-46f0-8716-0564f6b1f79b nodeName:}" failed. No retries permitted until 2026-07-25 00:40:29.890530439 +0000 UTC m=+252.279052820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xwds" (UID: "fc9f4877-ba9e-46f0-8716-0564f6b1f79b") : secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:29.847923 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:29.847890    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:40:29.850237 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:29.850211    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4c7b19e-0f50-493b-a39c-be861ced0055-metrics-certs\") pod \"network-metrics-daemon-bwhm2\" (UID: \"f4c7b19e-0f50-493b-a39c-be861ced0055\") " pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:40:29.869164 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:29.869140    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7xdtb\""
Jul 25 00:40:29.877797 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:29.877778    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhm2"
Jul 25 00:40:29.949244 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:29.949218    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:29.949364 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:29.949250    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:29.949718 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:29.949695    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:29.951728 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:29.951703    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc9f4877-ba9e-46f0-8716-0564f6b1f79b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwds\" (UID: \"fc9f4877-ba9e-46f0-8716-0564f6b1f79b\") " pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:29.993746 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:29.993722    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bwhm2"]
Jul 25 00:40:29.998670 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:40:29.996923    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c7b19e_0f50_493b_a39c_be861ced0055.slice/crio-cf46ad8eb9306ff1a7afc186ccfe5d4c88cab47970c6e368a65df4f4ca776b97 WatchSource:0}: Error finding container cf46ad8eb9306ff1a7afc186ccfe5d4c88cab47970c6e368a65df4f4ca776b97: Status 404 returned error can't find the container with id cf46ad8eb9306ff1a7afc186ccfe5d4c88cab47970c6e368a65df4f4ca776b97
Jul 25 00:40:30.152251 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:30.152228    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6x25j\""
Jul 25 00:40:30.160880 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:30.160864    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6xwds"
Jul 25 00:40:30.276840 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:30.276812    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6xwds"]
Jul 25 00:40:30.279504 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:40:30.279481    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9f4877_ba9e_46f0_8716_0564f6b1f79b.slice/crio-583c3c4fa5635e96a7817051c4da9dbd1743346a8f2bf097de554a045d57b609 WatchSource:0}: Error finding container 583c3c4fa5635e96a7817051c4da9dbd1743346a8f2bf097de554a045d57b609: Status 404 returned error can't find the container with id 583c3c4fa5635e96a7817051c4da9dbd1743346a8f2bf097de554a045d57b609
Jul 25 00:40:30.931377 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:30.931267    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xwds" event={"ID":"fc9f4877-ba9e-46f0-8716-0564f6b1f79b","Type":"ContainerStarted","Data":"583c3c4fa5635e96a7817051c4da9dbd1743346a8f2bf097de554a045d57b609"}
Jul 25 00:40:30.933000 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:30.932940    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bwhm2" event={"ID":"f4c7b19e-0f50-493b-a39c-be861ced0055","Type":"ContainerStarted","Data":"cf46ad8eb9306ff1a7afc186ccfe5d4c88cab47970c6e368a65df4f4ca776b97"}
Jul 25 00:40:31.939345 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:31.939302    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bwhm2" event={"ID":"f4c7b19e-0f50-493b-a39c-be861ced0055","Type":"ContainerStarted","Data":"291a6ece98d7282dab8c3c5f6110dde00b05c34165ebde873d083c2520a4ed59"}
Jul 25 00:40:31.939756 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:31.939394    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bwhm2" event={"ID":"f4c7b19e-0f50-493b-a39c-be861ced0055","Type":"ContainerStarted","Data":"3681625402da9ebf950b3366915d35f72bc1b796864f95af1b50d074626c05de"}
Jul 25 00:40:31.958465 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:31.958425    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bwhm2" podStartSLOduration=252.96387165 podStartE2EDuration="4m13.958412153s" totalImagesPullingTime="994.540503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:31.952575871 +0000 UTC m=+254.341098276" watchObservedRunningTime="2026-07-25 00:40:31.958412153 +0000 UTC m=+254.346934555"
Jul 25 00:40:35.953379 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:35.953343    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xwds" event={"ID":"fc9f4877-ba9e-46f0-8716-0564f6b1f79b","Type":"ContainerStarted","Data":"4e43049ee4046578a39139beff53a91ba3d16e195c6e2fc4823d07dc85e93a02"}
Jul 25 00:40:38.964922 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:38.964883    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xwds" event={"ID":"fc9f4877-ba9e-46f0-8716-0564f6b1f79b","Type":"ContainerStarted","Data":"ebe76f3dc0e7906ffeacda1da40c19a6a09ecfdfd400c8274f27818f2c2a3ab3"}
Jul 25 00:40:41.975484 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:41.975452    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xwds" event={"ID":"fc9f4877-ba9e-46f0-8716-0564f6b1f79b","Type":"ContainerStarted","Data":"07b9a4b3886c38a08ab343c85bdbf6d928cd7150dcec0ed76682f9bd284f66bd"}
Jul 25 00:40:41.994490 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:41.994442    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6xwds" podStartSLOduration=16.835514231 podStartE2EDuration="27.994425416s" totalImagesPullingTime="11.158911185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:40:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:41.993055336 +0000 UTC m=+264.381577738" watchObservedRunningTime="2026-07-25 00:40:41.994425416 +0000 UTC m=+264.382947882"
Jul 25 00:40:53.707610 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.707570    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"]
Jul 25 00:40:53.711316 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.711289    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.714050 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.714017    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wkmqm\""
Jul 25 00:40:53.714446 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.714412    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 25 00:40:53.714577 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.714420    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 25 00:40:53.714577 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.714465    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 25 00:40:53.715263 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.715236    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 25 00:40:53.715263 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.715255    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 25 00:40:53.719952 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.719931    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"]
Jul 25 00:40:53.723294 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.723266    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xrqns"]
Jul 25 00:40:53.728295 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.728264    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.730700 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.730676    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 25 00:40:53.730862 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.730833    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 25 00:40:53.730934 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.730833    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 25 00:40:53.730981 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.730945    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nql9s\""
Jul 25 00:40:53.814803 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.814778    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a0e2850-6e87-4536-ac5a-1218d2f32244-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.814905 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.814812    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-wtmp\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.814905 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.814835    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-accelerators-collector-config\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.814905 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.814894    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-textfile\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.815022 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.814940    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b48x\" (UniqueName: \"kubernetes.io/projected/86cb50a5-8a7f-4c7f-be32-8569e70be687-kube-api-access-6b48x\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.815022 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.815001    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a0e2850-6e87-4536-ac5a-1218d2f32244-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.815022 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.815017    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-tls\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.815152 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.815057    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86cb50a5-8a7f-4c7f-be32-8569e70be687-metrics-client-ca\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.815152 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.815099    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a0e2850-6e87-4536-ac5a-1218d2f32244-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.815152 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.815118    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/86cb50a5-8a7f-4c7f-be32-8569e70be687-root\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.815152 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.815139    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86cb50a5-8a7f-4c7f-be32-8569e70be687-sys\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.815278 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.815154    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29xp2\" (UniqueName: \"kubernetes.io/projected/5a0e2850-6e87-4536-ac5a-1218d2f32244-kube-api-access-29xp2\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.815278 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.815172    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915523 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915499    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a0e2850-6e87-4536-ac5a-1218d2f32244-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.915632 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915531    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-wtmp\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915632 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915553    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-accelerators-collector-config\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915632 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915571    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-textfile\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915632 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915587    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b48x\" (UniqueName: \"kubernetes.io/projected/86cb50a5-8a7f-4c7f-be32-8569e70be687-kube-api-access-6b48x\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915632 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915624    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a0e2850-6e87-4536-ac5a-1218d2f32244-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.915882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915646    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-tls\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915670    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86cb50a5-8a7f-4c7f-be32-8569e70be687-metrics-client-ca\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915695    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a0e2850-6e87-4536-ac5a-1218d2f32244-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.915882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915717    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/86cb50a5-8a7f-4c7f-be32-8569e70be687-root\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915743    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86cb50a5-8a7f-4c7f-be32-8569e70be687-sys\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915766    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29xp2\" (UniqueName: \"kubernetes.io/projected/5a0e2850-6e87-4536-ac5a-1218d2f32244-kube-api-access-29xp2\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.915882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915803    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-wtmp\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.915882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915860    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/86cb50a5-8a7f-4c7f-be32-8569e70be687-root\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.916387 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915951    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86cb50a5-8a7f-4c7f-be32-8569e70be687-sys\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.916387 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.915953    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-textfile\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.916490 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.916414    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86cb50a5-8a7f-4c7f-be32-8569e70be687-metrics-client-ca\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.916490 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.916418    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a0e2850-6e87-4536-ac5a-1218d2f32244-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.916676 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.916648    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-accelerators-collector-config\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.918495 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.918470    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a0e2850-6e87-4536-ac5a-1218d2f32244-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.918675 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.918650    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a0e2850-6e87-4536-ac5a-1218d2f32244-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:53.918803 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.918784    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-tls\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.918839 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.918819    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86cb50a5-8a7f-4c7f-be32-8569e70be687-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.924715 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.924691    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b48x\" (UniqueName: \"kubernetes.io/projected/86cb50a5-8a7f-4c7f-be32-8569e70be687-kube-api-access-6b48x\") pod \"node-exporter-xrqns\" (UID: \"86cb50a5-8a7f-4c7f-be32-8569e70be687\") " pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:53.925282 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:53.925256    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29xp2\" (UniqueName: \"kubernetes.io/projected/5a0e2850-6e87-4536-ac5a-1218d2f32244-kube-api-access-29xp2\") pod \"openshift-state-metrics-b8f49f5c6-7kbx8\" (UID: \"5a0e2850-6e87-4536-ac5a-1218d2f32244\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:54.022110 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:54.022038    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"
Jul 25 00:40:54.037326 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:54.037305    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xrqns"
Jul 25 00:40:54.046647 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:40:54.046627    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86cb50a5_8a7f_4c7f_be32_8569e70be687.slice/crio-37637ebe546dc3890cb4c143cd4a97151efe9bd0473f642ea2b33b27287ab82f WatchSource:0}: Error finding container 37637ebe546dc3890cb4c143cd4a97151efe9bd0473f642ea2b33b27287ab82f: Status 404 returned error can't find the container with id 37637ebe546dc3890cb4c143cd4a97151efe9bd0473f642ea2b33b27287ab82f
Jul 25 00:40:54.141823 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:54.141794    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8"]
Jul 25 00:40:54.144466 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:40:54.144441    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0e2850_6e87_4536_ac5a_1218d2f32244.slice/crio-88d633da925066e6af9b293e5b2eefc67da88245d00ef36430b9e11eb3baa10c WatchSource:0}: Error finding container 88d633da925066e6af9b293e5b2eefc67da88245d00ef36430b9e11eb3baa10c: Status 404 returned error can't find the container with id 88d633da925066e6af9b293e5b2eefc67da88245d00ef36430b9e11eb3baa10c
Jul 25 00:40:55.016965 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:55.016929    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8" event={"ID":"5a0e2850-6e87-4536-ac5a-1218d2f32244","Type":"ContainerStarted","Data":"b2a6625b99e18457fd5b6853a9e7cfc2aa132276c126a472bf7435577ebe91fc"}
Jul 25 00:40:55.017388 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:55.017002    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8" event={"ID":"5a0e2850-6e87-4536-ac5a-1218d2f32244","Type":"ContainerStarted","Data":"0fc2fb008139889ba334e6425bffee100ceee4d21509f97244b30b955d8eb27a"}
Jul 25 00:40:55.017388 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:55.017032    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8" event={"ID":"5a0e2850-6e87-4536-ac5a-1218d2f32244","Type":"ContainerStarted","Data":"88d633da925066e6af9b293e5b2eefc67da88245d00ef36430b9e11eb3baa10c"}
Jul 25 00:40:55.018255 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:55.018225    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xrqns" event={"ID":"86cb50a5-8a7f-4c7f-be32-8569e70be687","Type":"ContainerStarted","Data":"37637ebe546dc3890cb4c143cd4a97151efe9bd0473f642ea2b33b27287ab82f"}
Jul 25 00:40:56.023381 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:56.023355    2566 generic.go:356] "Generic (PLEG): container finished" podID="86cb50a5-8a7f-4c7f-be32-8569e70be687" containerID="126a889cf15e5a781220e24714044d997cd23a7f62fbd2f857fc92514176cda3" exitCode=0
Jul 25 00:40:56.023727 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:56.023451    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xrqns" event={"ID":"86cb50a5-8a7f-4c7f-be32-8569e70be687","Type":"ContainerDied","Data":"126a889cf15e5a781220e24714044d997cd23a7f62fbd2f857fc92514176cda3"}
Jul 25 00:40:56.025433 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:56.025408    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8" event={"ID":"5a0e2850-6e87-4536-ac5a-1218d2f32244","Type":"ContainerStarted","Data":"0a044dcead5829a85a2c54b3832373a47c501ca4f7747ee51b55a90afb9f3965"}
Jul 25 00:40:56.062598 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:56.062549    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7kbx8" podStartSLOduration=2.094898019 podStartE2EDuration="3.062535033s" totalImagesPullingTime="967.637014ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:56.062284309 +0000 UTC m=+278.450806712" watchObservedRunningTime="2026-07-25 00:40:56.062535033 +0000 UTC m=+278.451057436"
Jul 25 00:40:57.031202 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:57.031163    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xrqns" event={"ID":"86cb50a5-8a7f-4c7f-be32-8569e70be687","Type":"ContainerStarted","Data":"1fe516a5147122e8164082b2fc2ea1d0a65d375d81fc166bfd923a13950af49b"}
Jul 25 00:40:57.031644 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:57.031219    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xrqns" event={"ID":"86cb50a5-8a7f-4c7f-be32-8569e70be687","Type":"ContainerStarted","Data":"fbb45f41922e6166d961264c6a70fd4381d1cd477774f92faf28f8423f561327"}
Jul 25 00:40:57.055299 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:57.055244    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xrqns" podStartSLOduration=2.833869993 podStartE2EDuration="4.055229167s" totalImagesPullingTime="1.159292162s" totalInitContainerRuntime="62.067012ms" isStatefulPod=true podCreationTimestamp="2026-07-25 00:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:57.049294026 +0000 UTC m=+279.437816429" watchObservedRunningTime="2026-07-25 00:40:57.055229167 +0000 UTC m=+279.443751634"
Jul 25 00:40:58.028810 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.028777    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"]
Jul 25 00:40:58.031967 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.031938    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.034331 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.034254    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 25 00:40:58.034469 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.034344    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-tgtnr\""
Jul 25 00:40:58.034469 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.034344    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 25 00:40:58.035259 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.035238    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 25 00:40:58.035401 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.035245    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 25 00:40:58.035401 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.035392    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1eolal3ervlkg\""
Jul 25 00:40:58.038896 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.038856    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"]
Jul 25 00:40:58.146461 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.146433    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/84ee46b2-c55d-49c1-b9e8-64d4532efc86-secret-metrics-server-tls\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.146591 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.146486    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/84ee46b2-c55d-49c1-b9e8-64d4532efc86-metrics-server-audit-profiles\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.146591 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.146514    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/84ee46b2-c55d-49c1-b9e8-64d4532efc86-secret-metrics-server-client-certs\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.146683 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.146635    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84ee46b2-c55d-49c1-b9e8-64d4532efc86-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.146733 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.146682    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/84ee46b2-c55d-49c1-b9e8-64d4532efc86-audit-log\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.146784 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.146726    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zvhz\" (UniqueName: \"kubernetes.io/projected/84ee46b2-c55d-49c1-b9e8-64d4532efc86-kube-api-access-6zvhz\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.146784 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.146767    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84ee46b2-c55d-49c1-b9e8-64d4532efc86-client-ca-bundle\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.247197 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.247167    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84ee46b2-c55d-49c1-b9e8-64d4532efc86-client-ca-bundle\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.247326 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.247219    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/84ee46b2-c55d-49c1-b9e8-64d4532efc86-secret-metrics-server-tls\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.247326 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.247242    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/84ee46b2-c55d-49c1-b9e8-64d4532efc86-metrics-server-audit-profiles\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.247326 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.247268    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/84ee46b2-c55d-49c1-b9e8-64d4532efc86-secret-metrics-server-client-certs\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.247326 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.247311    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84ee46b2-c55d-49c1-b9e8-64d4532efc86-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.247504 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.247337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/84ee46b2-c55d-49c1-b9e8-64d4532efc86-audit-log\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.247504 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.247362    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zvhz\" (UniqueName: \"kubernetes.io/projected/84ee46b2-c55d-49c1-b9e8-64d4532efc86-kube-api-access-6zvhz\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.247818 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.247791    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/84ee46b2-c55d-49c1-b9e8-64d4532efc86-audit-log\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.248171 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.248145    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84ee46b2-c55d-49c1-b9e8-64d4532efc86-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.248337 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.248315    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/84ee46b2-c55d-49c1-b9e8-64d4532efc86-metrics-server-audit-profiles\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.250042 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.250018    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84ee46b2-c55d-49c1-b9e8-64d4532efc86-client-ca-bundle\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.250407 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.250389    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/84ee46b2-c55d-49c1-b9e8-64d4532efc86-secret-metrics-server-client-certs\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.250450 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.250416    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/84ee46b2-c55d-49c1-b9e8-64d4532efc86-secret-metrics-server-tls\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.256038 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.256013    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zvhz\" (UniqueName: \"kubernetes.io/projected/84ee46b2-c55d-49c1-b9e8-64d4532efc86-kube-api-access-6zvhz\") pod \"metrics-server-7bb68b47f5-bgjg8\" (UID: \"84ee46b2-c55d-49c1-b9e8-64d4532efc86\") " pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.342980 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.342918    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:40:58.467899 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:58.467872    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"]
Jul 25 00:40:58.472189 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:40:58.472163    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ee46b2_c55d_49c1_b9e8_64d4532efc86.slice/crio-de0c81fb4fad04f623098d3f5efa928ff45dcfda0a8bcbe807d4ce8250723505 WatchSource:0}: Error finding container de0c81fb4fad04f623098d3f5efa928ff45dcfda0a8bcbe807d4ce8250723505: Status 404 returned error can't find the container with id de0c81fb4fad04f623098d3f5efa928ff45dcfda0a8bcbe807d4ce8250723505
Jul 25 00:40:59.040035 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:40:59.040001    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8" event={"ID":"84ee46b2-c55d-49c1-b9e8-64d4532efc86","Type":"ContainerStarted","Data":"de0c81fb4fad04f623098d3f5efa928ff45dcfda0a8bcbe807d4ce8250723505"}
Jul 25 00:40:59.677174 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:40:59.677131    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-hw84r" podUID="e7839bc1-c826-40cd-9428-4e7080474366"
Jul 25 00:41:00.044151 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:00.044095    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hw84r"
Jul 25 00:41:00.044151 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:00.044119    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8" event={"ID":"84ee46b2-c55d-49c1-b9e8-64d4532efc86","Type":"ContainerStarted","Data":"0f5411670184933f317c7d3ce192f8d7581bb5315e58c2a8da44e6a142051418"}
Jul 25 00:41:00.062866 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:00.062815    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8" podStartSLOduration=0.723354956 podStartE2EDuration="2.06280367s" totalImagesPullingTime="1.339448714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:41:00.06127443 +0000 UTC m=+282.449796831" watchObservedRunningTime="2026-07-25 00:41:00.06280367 +0000 UTC m=+282.451326073"
Jul 25 00:41:02.979002 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:02.978963    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:41:02.981322 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:02.981296    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7839bc1-c826-40cd-9428-4e7080474366-metrics-tls\") pod \"dns-default-hw84r\" (UID: \"e7839bc1-c826-40cd-9428-4e7080474366\") " pod="openshift-dns/dns-default-hw84r"
Jul 25 00:41:03.047234 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:03.047209    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b2m9l\""
Jul 25 00:41:03.055849 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:03.055828    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hw84r"
Jul 25 00:41:03.178793 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:03.178758    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hw84r"]
Jul 25 00:41:03.182020 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:41:03.181998    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7839bc1_c826_40cd_9428_4e7080474366.slice/crio-9d31dfc4c4fe2bca899df166ba31351d57d9a30b6f31c6aabf5375763656ab48 WatchSource:0}: Error finding container 9d31dfc4c4fe2bca899df166ba31351d57d9a30b6f31c6aabf5375763656ab48: Status 404 returned error can't find the container with id 9d31dfc4c4fe2bca899df166ba31351d57d9a30b6f31c6aabf5375763656ab48
Jul 25 00:41:04.058448 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:04.058413    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hw84r" event={"ID":"e7839bc1-c826-40cd-9428-4e7080474366","Type":"ContainerStarted","Data":"9d31dfc4c4fe2bca899df166ba31351d57d9a30b6f31c6aabf5375763656ab48"}
Jul 25 00:41:05.062945 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:05.062888    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hw84r" event={"ID":"e7839bc1-c826-40cd-9428-4e7080474366","Type":"ContainerStarted","Data":"1f17ed512f2ec28f46be681564378a6aa993ef1e25ddd8393ce182eea1f1092e"}
Jul 25 00:41:05.062945 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:05.062935    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hw84r" event={"ID":"e7839bc1-c826-40cd-9428-4e7080474366","Type":"ContainerStarted","Data":"f8bda4f12babcde5238a710caf325915916a38eae29dfdd02616857a7c5219b9"}
Jul 25 00:41:05.063282 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:05.063008    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hw84r"
Jul 25 00:41:05.078213 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:05.078174    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hw84r" podStartSLOduration=251.466023728 podStartE2EDuration="4m13.078161254s" totalImagesPullingTime="1.612137526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:41:05.077050147 +0000 UTC m=+287.465572549" watchObservedRunningTime="2026-07-25 00:41:05.078161254 +0000 UTC m=+287.466683657"
Jul 25 00:41:13.060180 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:13.060138    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" podUID="063b0345-4b93-4e03-8c72-2bca6c04f2ab" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 25 00:41:15.068623 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:15.068589    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hw84r"
Jul 25 00:41:17.989710 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:17.989682    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 25 00:41:18.344005 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:18.343837    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:41:18.344005 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:18.343891    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:41:23.060341 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:23.060304    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" podUID="063b0345-4b93-4e03-8c72-2bca6c04f2ab" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 25 00:41:33.060446 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:33.060402    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" podUID="063b0345-4b93-4e03-8c72-2bca6c04f2ab" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 25 00:41:33.060840 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:33.060486    2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm"
Jul 25 00:41:33.060951 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:33.060914    2566 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"eefd2c99437a0d58bb347d8ea6a9f9d151ed9382c5f4918f3b4b4457f3127812"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 25 00:41:33.060991 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:33.060972    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" podUID="063b0345-4b93-4e03-8c72-2bca6c04f2ab" containerName="service-proxy" containerID="cri-o://eefd2c99437a0d58bb347d8ea6a9f9d151ed9382c5f4918f3b4b4457f3127812" gracePeriod=30
Jul 25 00:41:34.164566 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:34.164529    2566 generic.go:356] "Generic (PLEG): container finished" podID="063b0345-4b93-4e03-8c72-2bca6c04f2ab" containerID="eefd2c99437a0d58bb347d8ea6a9f9d151ed9382c5f4918f3b4b4457f3127812" exitCode=2
Jul 25 00:41:34.164942 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:34.164606    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" event={"ID":"063b0345-4b93-4e03-8c72-2bca6c04f2ab","Type":"ContainerDied","Data":"eefd2c99437a0d58bb347d8ea6a9f9d151ed9382c5f4918f3b4b4457f3127812"}
Jul 25 00:41:34.164942 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:34.164656    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cbd567f95-cn4lm" event={"ID":"063b0345-4b93-4e03-8c72-2bca6c04f2ab","Type":"ContainerStarted","Data":"2f95f55f3116cb9214f60474981b217b9f71a00b9deb5a47bacbbb2cb97c458b"}
Jul 25 00:41:38.348614 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:38.348578    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:41:38.352654 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:41:38.352622    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7bb68b47f5-bgjg8"
Jul 25 00:44:55.206749 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.206709    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qn776"]
Jul 25 00:44:55.208747 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.208727    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qn776"
Jul 25 00:44:55.210914 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.210887    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 25 00:44:55.211014 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.210951    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 25 00:44:55.211736 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.211711    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-b4stx\""
Jul 25 00:44:55.218986 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.218948    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qn776"]
Jul 25 00:44:55.355935 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.355906    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/660a04d8-2ad4-4a5f-9e07-3ec86b8cc392-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qn776\" (UID: \"660a04d8-2ad4-4a5f-9e07-3ec86b8cc392\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qn776"
Jul 25 00:44:55.356048 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.355976    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7m55\" (UniqueName: \"kubernetes.io/projected/660a04d8-2ad4-4a5f-9e07-3ec86b8cc392-kube-api-access-h7m55\") pod \"cert-manager-webhook-6486ffd997-qn776\" (UID: \"660a04d8-2ad4-4a5f-9e07-3ec86b8cc392\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qn776"
Jul 25 00:44:55.456505 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.456475    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/660a04d8-2ad4-4a5f-9e07-3ec86b8cc392-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qn776\" (UID: \"660a04d8-2ad4-4a5f-9e07-3ec86b8cc392\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qn776"
Jul 25 00:44:55.456635 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.456522    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7m55\" (UniqueName: \"kubernetes.io/projected/660a04d8-2ad4-4a5f-9e07-3ec86b8cc392-kube-api-access-h7m55\") pod \"cert-manager-webhook-6486ffd997-qn776\" (UID: \"660a04d8-2ad4-4a5f-9e07-3ec86b8cc392\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qn776"
Jul 25 00:44:55.464317 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.464250    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/660a04d8-2ad4-4a5f-9e07-3ec86b8cc392-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qn776\" (UID: \"660a04d8-2ad4-4a5f-9e07-3ec86b8cc392\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qn776"
Jul 25 00:44:55.464408 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.464387    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7m55\" (UniqueName: \"kubernetes.io/projected/660a04d8-2ad4-4a5f-9e07-3ec86b8cc392-kube-api-access-h7m55\") pod \"cert-manager-webhook-6486ffd997-qn776\" (UID: \"660a04d8-2ad4-4a5f-9e07-3ec86b8cc392\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qn776"
Jul 25 00:44:55.519260 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.519213    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qn776"
Jul 25 00:44:55.640750 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.640728    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qn776"]
Jul 25 00:44:55.643112 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:44:55.643084    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod660a04d8_2ad4_4a5f_9e07_3ec86b8cc392.slice/crio-029ce1521e937c009a11f96ea014852c218a95c6844bc67571f307ef30c16c4a WatchSource:0}: Error finding container 029ce1521e937c009a11f96ea014852c218a95c6844bc67571f307ef30c16c4a: Status 404 returned error can't find the container with id 029ce1521e937c009a11f96ea014852c218a95c6844bc67571f307ef30c16c4a
Jul 25 00:44:55.644985 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.644971    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:44:55.802752 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:55.802689    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qn776" event={"ID":"660a04d8-2ad4-4a5f-9e07-3ec86b8cc392","Type":"ContainerStarted","Data":"029ce1521e937c009a11f96ea014852c218a95c6844bc67571f307ef30c16c4a"}
Jul 25 00:44:58.814097 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:58.814003    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qn776" event={"ID":"660a04d8-2ad4-4a5f-9e07-3ec86b8cc392","Type":"ContainerStarted","Data":"32796240788e68ade577375ae9644b862f1468c360bcfb62d6b4627dc5566a86"}
Jul 25 00:44:58.814461 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:58.814221    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qn776"
Jul 25 00:44:58.828222 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:44:58.828177    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-qn776" podStartSLOduration=0.958941995 podStartE2EDuration="3.828163925s" totalImagesPullingTime="2.86922193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:44:58.827126341 +0000 UTC m=+521.215648742" watchObservedRunningTime="2026-07-25 00:44:58.828163925 +0000 UTC m=+521.216686326"
Jul 25 00:45:01.857243 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:01.857203    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"]
Jul 25 00:45:01.859374 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:01.859356    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"
Jul 25 00:45:01.861505 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:01.861474    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 25 00:45:01.862255 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:01.862228    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 25 00:45:01.862375 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:01.862339    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jbr8c\""
Jul 25 00:45:01.869510 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:01.869482    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"]
Jul 25 00:45:02.004882 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:02.004853    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d55a2a5f-a2cd-424c-8055-3627848c2cfe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pnx2s\" (UID: \"d55a2a5f-a2cd-424c-8055-3627848c2cfe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"
Jul 25 00:45:02.004993 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:02.004904    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4n5v\" (UniqueName: \"kubernetes.io/projected/d55a2a5f-a2cd-424c-8055-3627848c2cfe-kube-api-access-m4n5v\") pod \"openshift-lws-operator-fd8ccff4c-pnx2s\" (UID: \"d55a2a5f-a2cd-424c-8055-3627848c2cfe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"
Jul 25 00:45:02.105278 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:02.105248    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d55a2a5f-a2cd-424c-8055-3627848c2cfe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pnx2s\" (UID: \"d55a2a5f-a2cd-424c-8055-3627848c2cfe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"
Jul 25 00:45:02.105404 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:02.105318    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4n5v\" (UniqueName: \"kubernetes.io/projected/d55a2a5f-a2cd-424c-8055-3627848c2cfe-kube-api-access-m4n5v\") pod \"openshift-lws-operator-fd8ccff4c-pnx2s\" (UID: \"d55a2a5f-a2cd-424c-8055-3627848c2cfe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"
Jul 25 00:45:02.105670 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:02.105644    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d55a2a5f-a2cd-424c-8055-3627848c2cfe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pnx2s\" (UID: \"d55a2a5f-a2cd-424c-8055-3627848c2cfe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"
Jul 25 00:45:02.112857 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:02.112796    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4n5v\" (UniqueName: \"kubernetes.io/projected/d55a2a5f-a2cd-424c-8055-3627848c2cfe-kube-api-access-m4n5v\") pod \"openshift-lws-operator-fd8ccff4c-pnx2s\" (UID: \"d55a2a5f-a2cd-424c-8055-3627848c2cfe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"
Jul 25 00:45:02.168903 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:02.168874    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"
Jul 25 00:45:02.290472 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:02.290410    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s"]
Jul 25 00:45:02.293024 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:45:02.292999    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd55a2a5f_a2cd_424c_8055_3627848c2cfe.slice/crio-6f1592afc9bd0a472da1fdf3cfdb21b95428ce93f592d795ae5b0990abfb6ce5 WatchSource:0}: Error finding container 6f1592afc9bd0a472da1fdf3cfdb21b95428ce93f592d795ae5b0990abfb6ce5: Status 404 returned error can't find the container with id 6f1592afc9bd0a472da1fdf3cfdb21b95428ce93f592d795ae5b0990abfb6ce5
Jul 25 00:45:02.828408 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:02.828378    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s" event={"ID":"d55a2a5f-a2cd-424c-8055-3627848c2cfe","Type":"ContainerStarted","Data":"6f1592afc9bd0a472da1fdf3cfdb21b95428ce93f592d795ae5b0990abfb6ce5"}
Jul 25 00:45:04.819994 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:04.819966    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qn776"
Jul 25 00:45:06.843311 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:06.843278    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s" event={"ID":"d55a2a5f-a2cd-424c-8055-3627848c2cfe","Type":"ContainerStarted","Data":"7145925879d3c41a833ffd191355ce8d7295fbd8bc023bc4082a4ed77cb71100"}
Jul 25 00:45:06.860181 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:06.860141    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pnx2s" podStartSLOduration=2.214881205 podStartE2EDuration="5.860125014s" totalImagesPullingTime="3.645243809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:06.859966762 +0000 UTC m=+529.248489158" watchObservedRunningTime="2026-07-25 00:45:06.860125014 +0000 UTC m=+529.248647415"
Jul 25 00:45:22.566740 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.566700    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"]
Jul 25 00:45:22.639732 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.639702    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"]
Jul 25 00:45:22.639886 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.639793    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.643323 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.643301    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 25 00:45:22.643569 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.643552    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 25 00:45:22.643635 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.643592    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 25 00:45:22.643681 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.643661    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rcwf8\""
Jul 25 00:45:22.645019 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.644998    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 25 00:45:22.740150 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.740129    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b78e7779-41b6-4383-834a-1382d10f83f1-tmp\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.740241 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.740162    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b78e7779-41b6-4383-834a-1382d10f83f1-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.740241 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.740185    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b78e7779-41b6-4383-834a-1382d10f83f1-manifests\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.740241 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.740236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4x87\" (UniqueName: \"kubernetes.io/projected/b78e7779-41b6-4383-834a-1382d10f83f1-kube-api-access-x4x87\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.740338 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.740259    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b78e7779-41b6-4383-834a-1382d10f83f1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.840865 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.840811    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4x87\" (UniqueName: \"kubernetes.io/projected/b78e7779-41b6-4383-834a-1382d10f83f1-kube-api-access-x4x87\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.840865 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.840846    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b78e7779-41b6-4383-834a-1382d10f83f1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.841004 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.840873    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b78e7779-41b6-4383-834a-1382d10f83f1-tmp\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.841121 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.841084    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b78e7779-41b6-4383-834a-1382d10f83f1-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.841165 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.841126    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b78e7779-41b6-4383-834a-1382d10f83f1-manifests\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.841410 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.841393    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b78e7779-41b6-4383-834a-1382d10f83f1-manifests\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.843297 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.843277    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b78e7779-41b6-4383-834a-1382d10f83f1-tmp\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.843491 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.843469    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b78e7779-41b6-4383-834a-1382d10f83f1-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.843534 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.843474    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b78e7779-41b6-4383-834a-1382d10f83f1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.848974 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.848952    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4x87\" (UniqueName: \"kubernetes.io/projected/b78e7779-41b6-4383-834a-1382d10f83f1-kube-api-access-x4x87\") pod \"opendatahub-operator-controller-manager-9b8579fc9-dbl2k\" (UID: \"b78e7779-41b6-4383-834a-1382d10f83f1\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:22.949408 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:22.949385    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:23.088607 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:23.088578    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"]
Jul 25 00:45:23.092186 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:45:23.092128    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78e7779_41b6_4383_834a_1382d10f83f1.slice/crio-e8dd9ab62c3cb681bac6c3dc419c11d08c2981cd880e04335c481c024743c4b5 WatchSource:0}: Error finding container e8dd9ab62c3cb681bac6c3dc419c11d08c2981cd880e04335c481c024743c4b5: Status 404 returned error can't find the container with id e8dd9ab62c3cb681bac6c3dc419c11d08c2981cd880e04335c481c024743c4b5
Jul 25 00:45:23.902112 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:23.902083    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k" event={"ID":"b78e7779-41b6-4383-834a-1382d10f83f1","Type":"ContainerStarted","Data":"e8dd9ab62c3cb681bac6c3dc419c11d08c2981cd880e04335c481c024743c4b5"}
Jul 25 00:45:26.914223 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:26.914189    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k" event={"ID":"b78e7779-41b6-4383-834a-1382d10f83f1","Type":"ContainerStarted","Data":"1432e9422d470c3add29c04aab37afe4c1d759e1043c5922bae656b0067e0126"}
Jul 25 00:45:27.920085 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:27.920032    2566 generic.go:356] "Generic (PLEG): container finished" podID="b78e7779-41b6-4383-834a-1382d10f83f1" containerID="1432e9422d470c3add29c04aab37afe4c1d759e1043c5922bae656b0067e0126" exitCode=0
Jul 25 00:45:27.920430 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:27.920121    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k" event={"ID":"b78e7779-41b6-4383-834a-1382d10f83f1","Type":"ContainerDied","Data":"1432e9422d470c3add29c04aab37afe4c1d759e1043c5922bae656b0067e0126"}
Jul 25 00:45:28.926560 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:28.926521    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k" event={"ID":"b78e7779-41b6-4383-834a-1382d10f83f1","Type":"ContainerStarted","Data":"da6029b69dc978b3d162081f0680b169a0d6edc8b0054d80cbfe1ff5e70b537e"}
Jul 25 00:45:28.926933 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:28.926672    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:28.948478 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:28.948428    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k" podStartSLOduration=2.721161692 podStartE2EDuration="6.948414732s" totalImagesPullingTime="3.236930287s" totalInitContainerRuntime="990.322753ms" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:28.948163273 +0000 UTC m=+551.336685687" watchObservedRunningTime="2026-07-25 00:45:28.948414732 +0000 UTC m=+551.336937133"
Jul 25 00:45:39.933473 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:39.933438    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-dbl2k"
Jul 25 00:45:42.196267 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.196225    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mn5hf"]
Jul 25 00:45:42.199362 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.199341    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:45:42.201548 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.201515    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 25 00:45:42.201713 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.201660    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-b5x4l\""
Jul 25 00:45:42.211348 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.211320    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mn5hf"]
Jul 25 00:45:42.268912 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.268887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5pkb\" (UniqueName: \"kubernetes.io/projected/250dc570-ac6d-45e0-956d-ffa2996315d1-kube-api-access-r5pkb\") pod \"odh-model-controller-7998c67dc4-mn5hf\" (UID: \"250dc570-ac6d-45e0-956d-ffa2996315d1\") " pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:45:42.269016 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.268916    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/250dc570-ac6d-45e0-956d-ffa2996315d1-cert\") pod \"odh-model-controller-7998c67dc4-mn5hf\" (UID: \"250dc570-ac6d-45e0-956d-ffa2996315d1\") " pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:45:42.369975 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.369948    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5pkb\" (UniqueName: \"kubernetes.io/projected/250dc570-ac6d-45e0-956d-ffa2996315d1-kube-api-access-r5pkb\") pod \"odh-model-controller-7998c67dc4-mn5hf\" (UID: \"250dc570-ac6d-45e0-956d-ffa2996315d1\") " pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:45:42.370106 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.369980    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/250dc570-ac6d-45e0-956d-ffa2996315d1-cert\") pod \"odh-model-controller-7998c67dc4-mn5hf\" (UID: \"250dc570-ac6d-45e0-956d-ffa2996315d1\") " pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:45:42.370184 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:45:42.370160    2566 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 25 00:45:42.370258 ip-10-0-140-134 kubenswrapper[2566]: E0725 00:45:42.370242    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/250dc570-ac6d-45e0-956d-ffa2996315d1-cert podName:250dc570-ac6d-45e0-956d-ffa2996315d1 nodeName:}" failed. No retries permitted until 2026-07-25 00:45:42.870219744 +0000 UTC m=+565.258742130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/250dc570-ac6d-45e0-956d-ffa2996315d1-cert") pod "odh-model-controller-7998c67dc4-mn5hf" (UID: "250dc570-ac6d-45e0-956d-ffa2996315d1") : secret "odh-model-controller-webhook-cert" not found
Jul 25 00:45:42.382748 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.382714    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5pkb\" (UniqueName: \"kubernetes.io/projected/250dc570-ac6d-45e0-956d-ffa2996315d1-kube-api-access-r5pkb\") pod \"odh-model-controller-7998c67dc4-mn5hf\" (UID: \"250dc570-ac6d-45e0-956d-ffa2996315d1\") " pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:45:42.874541 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.874514    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/250dc570-ac6d-45e0-956d-ffa2996315d1-cert\") pod \"odh-model-controller-7998c67dc4-mn5hf\" (UID: \"250dc570-ac6d-45e0-956d-ffa2996315d1\") " pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:45:42.876943 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:42.876904    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/250dc570-ac6d-45e0-956d-ffa2996315d1-cert\") pod \"odh-model-controller-7998c67dc4-mn5hf\" (UID: \"250dc570-ac6d-45e0-956d-ffa2996315d1\") " pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:45:43.109981 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:43.109951    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:45:43.234310 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:43.234287    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mn5hf"]
Jul 25 00:45:43.236484 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:45:43.236461    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod250dc570_ac6d_45e0_956d_ffa2996315d1.slice/crio-e364f1989a63bfe3ed24693f5fec7dae98f69d34d8957bf8d7d1be9bbc0f2cbf WatchSource:0}: Error finding container e364f1989a63bfe3ed24693f5fec7dae98f69d34d8957bf8d7d1be9bbc0f2cbf: Status 404 returned error can't find the container with id e364f1989a63bfe3ed24693f5fec7dae98f69d34d8957bf8d7d1be9bbc0f2cbf
Jul 25 00:45:43.979689 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:43.979654    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf" event={"ID":"250dc570-ac6d-45e0-956d-ffa2996315d1","Type":"ContainerStarted","Data":"e364f1989a63bfe3ed24693f5fec7dae98f69d34d8957bf8d7d1be9bbc0f2cbf"}
Jul 25 00:45:45.929484 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.929444    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"]
Jul 25 00:45:45.933079 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.933036    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:45.935300 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.935274    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7sgxr\""
Jul 25 00:45:45.936134 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.936109    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 25 00:45:45.936266 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.936249    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 25 00:45:45.936324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.936304    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 25 00:45:45.942838 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.942817    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"]
Jul 25 00:45:45.996634 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.996610    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/afc8c728-47f9-41f2-9181-c814c3f0ec52-metrics-cert\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:45.996741 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.996642    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/afc8c728-47f9-41f2-9181-c814c3f0ec52-cert\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:45.996741 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.996660    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2zhz\" (UniqueName: \"kubernetes.io/projected/afc8c728-47f9-41f2-9181-c814c3f0ec52-kube-api-access-z2zhz\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:45.996848 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:45.996782    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/afc8c728-47f9-41f2-9181-c814c3f0ec52-manager-config\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:46.097461 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.097435    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/afc8c728-47f9-41f2-9181-c814c3f0ec52-metrics-cert\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:46.097574 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.097464    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/afc8c728-47f9-41f2-9181-c814c3f0ec52-cert\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:46.097574 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.097481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2zhz\" (UniqueName: \"kubernetes.io/projected/afc8c728-47f9-41f2-9181-c814c3f0ec52-kube-api-access-z2zhz\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:46.097574 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.097515    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/afc8c728-47f9-41f2-9181-c814c3f0ec52-manager-config\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:46.098128 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.098106    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/afc8c728-47f9-41f2-9181-c814c3f0ec52-manager-config\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:46.099837 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.099818    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/afc8c728-47f9-41f2-9181-c814c3f0ec52-metrics-cert\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:46.099921 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.099846    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/afc8c728-47f9-41f2-9181-c814c3f0ec52-cert\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:46.112784 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.112758    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2zhz\" (UniqueName: \"kubernetes.io/projected/afc8c728-47f9-41f2-9181-c814c3f0ec52-kube-api-access-z2zhz\") pod \"lws-controller-manager-7bd5dcbf9d-bql5c\" (UID: \"afc8c728-47f9-41f2-9181-c814c3f0ec52\") " pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:46.244519 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.244465    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:46.377366 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.377343    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"]
Jul 25 00:45:46.379710 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:45:46.379688    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc8c728_47f9_41f2_9181_c814c3f0ec52.slice/crio-b4d4ebd651495d9c7bdef0be5823a1dbd8019f8fba7bdfd54feb5cabc36bfbe2 WatchSource:0}: Error finding container b4d4ebd651495d9c7bdef0be5823a1dbd8019f8fba7bdfd54feb5cabc36bfbe2: Status 404 returned error can't find the container with id b4d4ebd651495d9c7bdef0be5823a1dbd8019f8fba7bdfd54feb5cabc36bfbe2
Jul 25 00:45:46.992270 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.992232    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf" event={"ID":"250dc570-ac6d-45e0-956d-ffa2996315d1","Type":"ContainerStarted","Data":"767626e89f69f6c32ce5f2e90d5a8b0ab609ee274e8488b3bb484da1f0e10c47"}
Jul 25 00:45:46.992710 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.992347    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:45:46.993510 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:46.993481    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c" event={"ID":"afc8c728-47f9-41f2-9181-c814c3f0ec52","Type":"ContainerStarted","Data":"b4d4ebd651495d9c7bdef0be5823a1dbd8019f8fba7bdfd54feb5cabc36bfbe2"}
Jul 25 00:45:47.009051 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:47.008995    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf" podStartSLOduration=2.232087871 podStartE2EDuration="5.008978145s" totalImagesPullingTime="2.776890274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:47.006619705 +0000 UTC m=+569.395142108" watchObservedRunningTime="2026-07-25 00:45:47.008978145 +0000 UTC m=+569.397500549"
Jul 25 00:45:50.801272 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.801233    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"]
Jul 25 00:45:50.804342 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.804327    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:50.806595 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.806561    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 25 00:45:50.806595 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.806584    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 25 00:45:50.806761 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.806612    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ms2k8\""
Jul 25 00:45:50.806761 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.806584    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 25 00:45:50.806761 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.806663    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 25 00:45:50.806761 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.806692    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 25 00:45:50.807252 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.807237    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 25 00:45:50.816585 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.816566    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"]
Jul 25 00:45:50.927813 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.927787    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:50.927911 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.927818    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:50.927911 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.927843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d9483e9-b41a-477b-8e82-9c1310d22c01-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:50.927911 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.927859    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x248\" (UniqueName: \"kubernetes.io/projected/7d9483e9-b41a-477b-8e82-9c1310d22c01-kube-api-access-9x248\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:50.927911 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.927896    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d9483e9-b41a-477b-8e82-9c1310d22c01-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:50.928036 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.927995    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:50.928036 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:50.928025    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.028876 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.028854    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.028973 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.028881    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.028973 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.028903    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.028973 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.028922    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.028973 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.028962    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d9483e9-b41a-477b-8e82-9c1310d22c01-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.029209 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.029193    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x248\" (UniqueName: \"kubernetes.io/projected/7d9483e9-b41a-477b-8e82-9c1310d22c01-kube-api-access-9x248\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.029271 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.029242    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d9483e9-b41a-477b-8e82-9c1310d22c01-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.029540 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.029509    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.031505 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.031484    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.031505 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.031494    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.031631 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.031516    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d9483e9-b41a-477b-8e82-9c1310d22c01-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.031711 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.031689    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d9483e9-b41a-477b-8e82-9c1310d22c01-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.042620 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.042588    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x248\" (UniqueName: \"kubernetes.io/projected/7d9483e9-b41a-477b-8e82-9c1310d22c01-kube-api-access-9x248\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.042714 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.042694    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d9483e9-b41a-477b-8e82-9c1310d22c01-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-np2wj\" (UID: \"7d9483e9-b41a-477b-8e82-9c1310d22c01\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.114095 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.114038    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:51.239214 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:51.239051    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"]
Jul 25 00:45:51.241305 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:45:51.241273    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9483e9_b41a_477b_8e82_9c1310d22c01.slice/crio-5c5637d0c2494c30fc94b595b1489140b5081f4d4cc69289b9736ff57da21de8 WatchSource:0}: Error finding container 5c5637d0c2494c30fc94b595b1489140b5081f4d4cc69289b9736ff57da21de8: Status 404 returned error can't find the container with id 5c5637d0c2494c30fc94b595b1489140b5081f4d4cc69289b9736ff57da21de8
Jul 25 00:45:52.018384 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:52.018356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj" event={"ID":"7d9483e9-b41a-477b-8e82-9c1310d22c01","Type":"ContainerStarted","Data":"5c5637d0c2494c30fc94b595b1489140b5081f4d4cc69289b9736ff57da21de8"}
Jul 25 00:45:55.032950 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:55.032912    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c" event={"ID":"afc8c728-47f9-41f2-9181-c814c3f0ec52","Type":"ContainerStarted","Data":"c9b026ee684d051edbe92a34659ab9bf8e9929226f9efbe8580e729522832214"}
Jul 25 00:45:55.033399 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:55.033019    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:45:55.049779 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:55.049739    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c" podStartSLOduration=2.198958987 podStartE2EDuration="10.049726611s" totalImagesPullingTime="7.850767624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:55.049113169 +0000 UTC m=+577.437635572" watchObservedRunningTime="2026-07-25 00:45:55.049726611 +0000 UTC m=+577.438249014"
Jul 25 00:45:55.945538 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:55.945500    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 25 00:45:55.945676 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:55.945584    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 25 00:45:56.038630 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:56.038598    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj" event={"ID":"7d9483e9-b41a-477b-8e82-9c1310d22c01","Type":"ContainerStarted","Data":"428ff6009efc5adc43b0b43214690055f30a3bd84e826f1aaf176eb887fb23ab"}
Jul 25 00:45:56.039016 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:56.038816    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:45:56.040251 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:56.040226    2566 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-np2wj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.18:8080/ready\": dial tcp 10.134.0.18:8080: connect: connection refused" start-of-body=
Jul 25 00:45:56.040342 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:56.040282    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj" podUID="7d9483e9-b41a-477b-8e82-9c1310d22c01" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.18:8080/ready\": dial tcp 10.134.0.18:8080: connect: connection refused"
Jul 25 00:45:56.059098 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:56.059040    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj" podStartSLOduration=1.357230816 podStartE2EDuration="6.059027069s" totalImagesPullingTime="4.701796253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:56.057962193 +0000 UTC m=+578.446484597" watchObservedRunningTime="2026-07-25 00:45:56.059027069 +0000 UTC m=+578.447549470"
Jul 25 00:45:57.043714 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:57.043685    2566 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-np2wj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 25 00:45:57.044046 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:57.043736    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj" podUID="7d9483e9-b41a-477b-8e82-9c1310d22c01" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 25 00:45:57.999658 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:45:57.999627    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mn5hf"
Jul 25 00:46:00.043681 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:46:00.043655    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-np2wj"
Jul 25 00:46:06.040450 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:46:06.040408    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7bd5dcbf9d-bql5c"
Jul 25 00:48:30.278981 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.278944    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:48:30.282027 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.282006    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"
Jul 25 00:48:30.284184 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.284162    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 25 00:48:30.284310 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.284287    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 25 00:48:30.284371 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.284292    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 25 00:48:30.284997 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.284983    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9qz72\""
Jul 25 00:48:30.290127 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.290104    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:48:30.323771 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.323753    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:48:30.417739 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.417714    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19385261-42c9-494d-8847-3e80717fe0c5-config-file\") pod \"limitador-limitador-54d6d988d5-sqmmx\" (UID: \"19385261-42c9-494d-8847-3e80717fe0c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"
Jul 25 00:48:30.417830 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.417755    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjxxf\" (UniqueName: \"kubernetes.io/projected/19385261-42c9-494d-8847-3e80717fe0c5-kube-api-access-vjxxf\") pod \"limitador-limitador-54d6d988d5-sqmmx\" (UID: \"19385261-42c9-494d-8847-3e80717fe0c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"
Jul 25 00:48:30.518584 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.518549    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjxxf\" (UniqueName: \"kubernetes.io/projected/19385261-42c9-494d-8847-3e80717fe0c5-kube-api-access-vjxxf\") pod \"limitador-limitador-54d6d988d5-sqmmx\" (UID: \"19385261-42c9-494d-8847-3e80717fe0c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"
Jul 25 00:48:30.518682 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.518621    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19385261-42c9-494d-8847-3e80717fe0c5-config-file\") pod \"limitador-limitador-54d6d988d5-sqmmx\" (UID: \"19385261-42c9-494d-8847-3e80717fe0c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"
Jul 25 00:48:30.519175 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.519157    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19385261-42c9-494d-8847-3e80717fe0c5-config-file\") pod \"limitador-limitador-54d6d988d5-sqmmx\" (UID: \"19385261-42c9-494d-8847-3e80717fe0c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"
Jul 25 00:48:30.525888 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.525864    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjxxf\" (UniqueName: \"kubernetes.io/projected/19385261-42c9-494d-8847-3e80717fe0c5-kube-api-access-vjxxf\") pod \"limitador-limitador-54d6d988d5-sqmmx\" (UID: \"19385261-42c9-494d-8847-3e80717fe0c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"
Jul 25 00:48:30.593324 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.593265    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"
Jul 25 00:48:30.716331 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:30.716302    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:48:30.719380 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:48:30.719354    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19385261_42c9_494d_8847_3e80717fe0c5.slice/crio-c48b50cba792063910d946d5cc08f162e670b5113ae2f20992fb669292bacec0 WatchSource:0}: Error finding container c48b50cba792063910d946d5cc08f162e670b5113ae2f20992fb669292bacec0: Status 404 returned error can't find the container with id c48b50cba792063910d946d5cc08f162e670b5113ae2f20992fb669292bacec0
Jul 25 00:48:31.660853 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:31.660815    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx" event={"ID":"19385261-42c9-494d-8847-3e80717fe0c5","Type":"ContainerStarted","Data":"c48b50cba792063910d946d5cc08f162e670b5113ae2f20992fb669292bacec0"}
Jul 25 00:48:33.674126 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:33.674060    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx" event={"ID":"19385261-42c9-494d-8847-3e80717fe0c5","Type":"ContainerStarted","Data":"0853706ac3dd3cfc5c1984c7a7f16fc6497e85e204bf854008c3f5ff3a5814e6"}
Jul 25 00:48:33.674496 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:33.674215    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"
Jul 25 00:48:33.690185 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:33.690127    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx" podStartSLOduration=1.364692847 podStartE2EDuration="3.69011164s" totalImagesPullingTime="2.325418793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:48:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:48:33.688258622 +0000 UTC m=+736.076781024" watchObservedRunningTime="2026-07-25 00:48:33.69011164 +0000 UTC m=+736.078634042"
Jul 25 00:48:44.680119 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:48:44.680079    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"
Jul 25 00:49:41.710548 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:49:41.710475    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:50:12.689178 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:12.689145    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:50:23.979219 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:23.979173    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:50:32.784562 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:32.784523    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:50:41.784764 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.784722    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"]
Jul 25 00:50:41.787978 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.787958    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.790297 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.790267    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gjm4r\""
Jul 25 00:50:41.790443 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.790302    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 25 00:50:41.790920 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.790890    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 25 00:50:41.791046 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.790959    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 25 00:50:41.798418 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.798395    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"]
Jul 25 00:50:41.821211 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.821188    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wv4c\" (UniqueName: \"kubernetes.io/projected/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-kube-api-access-2wv4c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.821314 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.821216    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.821314 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.821236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.821314 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.821263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.821414 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.821338    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.821414 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.821374    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.921611 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.921574    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.921755 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.921621    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.921755 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.921664    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.921755 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.921694    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wv4c\" (UniqueName: \"kubernetes.io/projected/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-kube-api-access-2wv4c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.921755 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.921711    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.921755 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.921729    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.922027 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.922005    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.922132 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.922110    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.922196 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.922147    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.924153 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.924125    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.924542 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.924516    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:41.928762 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:41.928742    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wv4c\" (UniqueName: \"kubernetes.io/projected/46c22c91-e51b-4dcc-9b8e-ef7c245c6cce-kube-api-access-2wv4c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs\" (UID: \"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:42.099304 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:42.099245    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:42.220741 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:42.220666    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"]
Jul 25 00:50:42.222867 ip-10-0-140-134 kubenswrapper[2566]: W0725 00:50:42.222841    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c22c91_e51b_4dcc_9b8e_ef7c245c6cce.slice/crio-53970490778e6aab14dddd011699d9419bc741f23a805ca9b9f1a5d576970d46 WatchSource:0}: Error finding container 53970490778e6aab14dddd011699d9419bc741f23a805ca9b9f1a5d576970d46: Status 404 returned error can't find the container with id 53970490778e6aab14dddd011699d9419bc741f23a805ca9b9f1a5d576970d46
Jul 25 00:50:42.224200 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:42.224175    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:50:43.213985 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:43.213953    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs" event={"ID":"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce","Type":"ContainerStarted","Data":"ea96dbb8500d25d840d1f6dfb29d09af4f28330679e319d7d0187c307197a9d7"}
Jul 25 00:50:43.214289 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:43.214003    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs" event={"ID":"46c22c91-e51b-4dcc-9b8e-ef7c245c6cce","Type":"ContainerStarted","Data":"53970490778e6aab14dddd011699d9419bc741f23a805ca9b9f1a5d576970d46"}
Jul 25 00:50:43.229008 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:43.228963    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs" podStartSLOduration=1.323864126 podStartE2EDuration="2.228945198s" totalImagesPullingTime="905.081072ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:50:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:50:43.228342541 +0000 UTC m=+865.616864946" watchObservedRunningTime="2026-07-25 00:50:43.228945198 +0000 UTC m=+865.617467601"
Jul 25 00:50:43.484706 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:43.484639    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:50:52.100175 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:52.100138    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:52.100574 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:52.100476    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:52.112048 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:52.112021    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:50:52.260853 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:50:52.260829    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs"
Jul 25 00:51:00.936141 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:51:00.936097    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:51:26.890968 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:51:26.890921    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:52:52.792578 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:52:52.792495    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:53:01.991832 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:53:01.991788    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:53:12.583662 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:53:12.583613    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:54:58.592128 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:54:58.592084    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:55:08.188464 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:55:08.188428    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:55:36.681428 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:55:36.681385    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:55:46.383179 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:55:46.383094    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:55:55.686909 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:55:55.686863    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:56:07.183672 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:56:07.183632    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:56:18.191895 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:56:18.191866    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:56:29.187806 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:56:29.187766    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:56:40.584014 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:56:40.583967    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:56:50.483571 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:56:50.483528    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:56:56.482479 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:56:56.482434    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:57:04.188502 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:57:04.188453    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:57:16.082767 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:57:16.082721    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:57:28.082869 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:57:28.082829    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:57:50.988266 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:57:50.988224    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:58:01.586405 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:58:01.586363    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:58:17.191726 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:58:17.191686    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:58:28.287675 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:58:28.287627    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:58:38.083419 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:58:38.083369    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:58:49.287969 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:58:49.287880    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 00:59:02.085995 ip-10-0-140-134 kubenswrapper[2566]: I0725 00:59:02.085950    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:01:18.384353 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:01:18.384318    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:01:27.886578 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:01:27.886544    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:01:40.684354 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:01:40.684267    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:01:51.487630 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:01:51.487592    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:02:03.288364 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:02:03.288329    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:02:13.281698 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:02:13.281659    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:02:22.383533 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:02:22.383497    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:02:33.686884 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:02:33.686846    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:02:45.080540 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:02:45.080504    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:02:52.983210 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:02:52.983171    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:03:04.183347 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:03:04.183310    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:03:28.781411 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:03:28.781376    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:03:36.802864 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:03:36.802823    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:04:05.100380 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.100342    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"]
Jul 25 01:04:05.103360 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.103341    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.105563 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.105532    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-b59a4a38-kserve-self-signed-certs\""
Jul 25 01:04:05.118477 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.118443    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"]
Jul 25 01:04:05.230434 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.230388    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tmp-dir\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.230619 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.230483    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-model-cache\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.230619 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.230518    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btb6v\" (UniqueName: \"kubernetes.io/projected/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-kube-api-access-btb6v\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.230619 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.230549    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-home\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.230745 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.230674    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-dshm\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.230777 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.230760    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tls-certs\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.331432 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.331378    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tmp-dir\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.331432 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.331443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-model-cache\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.331432 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.331460    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btb6v\" (UniqueName: \"kubernetes.io/projected/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-kube-api-access-btb6v\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.331745 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.331490    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-home\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.331745 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.331528    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-dshm\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.331745 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.331579    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tls-certs\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.331896 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.331868    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tmp-dir\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.331961 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.331897    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-home\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.331961 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.331944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-model-cache\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.333803 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.333782    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-dshm\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.334053 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.334036    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tls-certs\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.339512 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.339486    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btb6v\" (UniqueName: \"kubernetes.io/projected/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-kube-api-access-btb6v\") pod \"e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") " pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.412569 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.412526    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:05.548650 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:05.548626    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"]
Jul 25 01:04:05.551180 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:04:05.551149    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c9803b1_313e_47a6_9b41_9ef4a39c5c01.slice/crio-95b1f7515ce498db1a54acec944c05332569c7c87f41a57cb5a1d60d6c93b1aa WatchSource:0}: Error finding container 95b1f7515ce498db1a54acec944c05332569c7c87f41a57cb5a1d60d6c93b1aa: Status 404 returned error can't find the container with id 95b1f7515ce498db1a54acec944c05332569c7c87f41a57cb5a1d60d6c93b1aa
Jul 25 01:04:06.500749 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:06.500710    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw" event={"ID":"8c9803b1-313e-47a6-9b41-9ef4a39c5c01","Type":"ContainerStarted","Data":"5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194"}
Jul 25 01:04:06.500749 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:06.500752    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw" event={"ID":"8c9803b1-313e-47a6-9b41-9ef4a39c5c01","Type":"ContainerStarted","Data":"95b1f7515ce498db1a54acec944c05332569c7c87f41a57cb5a1d60d6c93b1aa"}
Jul 25 01:04:06.519174 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:06.519120    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw" podStartSLOduration=1.519102859 podStartE2EDuration="1.519102859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:04:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:04:06.517821783 +0000 UTC m=+1668.906344187" watchObservedRunningTime="2026-07-25 01:04:06.519102859 +0000 UTC m=+1668.907625262"
Jul 25 01:04:08.396749 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:08.396712    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:04:15.413502 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:15.413393    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:15.413502 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:15.413461    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:15.426145 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:15.426111    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:15.555051 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:15.555022    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:04:36.690465 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:04:36.690428    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:05:20.305124 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:20.305056    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"]
Jul 25 01:05:22.794108 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:22.794054    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:05:35.323300 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.323248    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw" podUID="8c9803b1-313e-47a6-9b41-9ef4a39c5c01" containerName="main" containerID="cri-o://5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194" gracePeriod=45
Jul 25 01:05:35.567033 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.567011    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:05:35.663689 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.663657    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btb6v\" (UniqueName: \"kubernetes.io/projected/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-kube-api-access-btb6v\") pod \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") "
Jul 25 01:05:35.663848 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.663697    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-model-cache\") pod \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") "
Jul 25 01:05:35.663848 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.663746    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tmp-dir\") pod \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") "
Jul 25 01:05:35.663949 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.663920    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-dshm\") pod \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") "
Jul 25 01:05:35.664002 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.663962    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8c9803b1-313e-47a6-9b41-9ef4a39c5c01" (UID: "8c9803b1-313e-47a6-9b41-9ef4a39c5c01"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:05:35.664002 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.663971    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tls-certs\") pod \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") "
Jul 25 01:05:35.664002 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.663980    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-model-cache" (OuterVolumeSpecName: "model-cache") pod "8c9803b1-313e-47a6-9b41-9ef4a39c5c01" (UID: "8c9803b1-313e-47a6-9b41-9ef4a39c5c01"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:05:35.664227 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.664003    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-home\") pod \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\" (UID: \"8c9803b1-313e-47a6-9b41-9ef4a39c5c01\") "
Jul 25 01:05:35.664227 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.664200    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-home" (OuterVolumeSpecName: "home") pod "8c9803b1-313e-47a6-9b41-9ef4a39c5c01" (UID: "8c9803b1-313e-47a6-9b41-9ef4a39c5c01"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:05:35.664309 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.664297    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:05:35.664343 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.664317    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:05:35.664343 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.664332    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:05:35.665986 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.665959    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8c9803b1-313e-47a6-9b41-9ef4a39c5c01" (UID: "8c9803b1-313e-47a6-9b41-9ef4a39c5c01"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:05:35.666107 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.665987    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-dshm" (OuterVolumeSpecName: "dshm") pod "8c9803b1-313e-47a6-9b41-9ef4a39c5c01" (UID: "8c9803b1-313e-47a6-9b41-9ef4a39c5c01"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:05:35.666107 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.666030    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-kube-api-access-btb6v" (OuterVolumeSpecName: "kube-api-access-btb6v") pod "8c9803b1-313e-47a6-9b41-9ef4a39c5c01" (UID: "8c9803b1-313e-47a6-9b41-9ef4a39c5c01"). InnerVolumeSpecName "kube-api-access-btb6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:05:35.765287 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.765260    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:05:35.765287 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.765281    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:05:35.765287 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.765291    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btb6v\" (UniqueName: \"kubernetes.io/projected/8c9803b1-313e-47a6-9b41-9ef4a39c5c01-kube-api-access-btb6v\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:05:35.878367 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.878339    2566 generic.go:356] "Generic (PLEG): container finished" podID="8c9803b1-313e-47a6-9b41-9ef4a39c5c01" containerID="5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194" exitCode=0
Jul 25 01:05:35.878467 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.878384    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"
Jul 25 01:05:35.878467 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.878425    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw" event={"ID":"8c9803b1-313e-47a6-9b41-9ef4a39c5c01","Type":"ContainerDied","Data":"5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194"}
Jul 25 01:05:35.878536 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.878469    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw" event={"ID":"8c9803b1-313e-47a6-9b41-9ef4a39c5c01","Type":"ContainerDied","Data":"95b1f7515ce498db1a54acec944c05332569c7c87f41a57cb5a1d60d6c93b1aa"}
Jul 25 01:05:35.878536 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.878498    2566 scope.go:117] "RemoveContainer" containerID="5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194"
Jul 25 01:05:35.887666 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.887651    2566 scope.go:117] "RemoveContainer" containerID="5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194"
Jul 25 01:05:35.887904 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:05:35.887887    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194\": container with ID starting with 5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194 not found: ID does not exist" containerID="5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194"
Jul 25 01:05:35.887973 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.887914    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194"} err="failed to get container status \"5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194\": rpc error: code = NotFound desc = could not find container \"5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194\": container with ID starting with 5569e535e17c2d71632ea75b5dd8de783485b65630e9a70d3cfd12d6080de194 not found: ID does not exist"
Jul 25 01:05:35.899101 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.899059    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"]
Jul 25 01:05:35.902223 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:35.902202    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw"]
Jul 25 01:05:36.171174 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:36.171150    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9803b1-313e-47a6-9b41-9ef4a39c5c01" path="/var/lib/kubelet/pods/8c9803b1-313e-47a6-9b41-9ef4a39c5c01/volumes"
Jul 25 01:05:38.496930 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:38.496896    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:05:40.545213 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:40.545169    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-b59a4a38-kserve-56db6b467f-p2jhw" podUID="8c9803b1-313e-47a6-9b41-9ef4a39c5c01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.21:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 25 01:05:45.386341 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:05:45.386309    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:06:02.603767 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:06:02.603703    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:06:12.490845 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:06:12.490772    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:06:22.090873 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:06:22.090840    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:06:32.787358 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:06:32.787324    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:06:41.992561 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:06:41.992525    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:06:51.688141 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:06:51.688103    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:07:05.381998 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:07:05.381957    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:07:14.794455 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:07:14.794415    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:07:24.684096 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:07:24.684050    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:07:37.690508 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:07:37.690462    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:07:48.990092 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:07:48.990001    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:07:57.991130 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:07:57.991093    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:08:09.900315 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:08:09.900276    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:08:22.285618 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:08:22.285572    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:08:34.094508 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:08:34.094475    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:08:46.889173 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:08:46.889139    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:09:02.085539 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:02.085500    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:09:17.199767 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:17.199729    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:09:36.146036 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.146003    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"]
Jul 25 01:09:36.146512 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.146307    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9803b1-313e-47a6-9b41-9ef4a39c5c01" containerName="main"
Jul 25 01:09:36.146512 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.146319    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9803b1-313e-47a6-9b41-9ef4a39c5c01" containerName="main"
Jul 25 01:09:36.146512 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.146372    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9803b1-313e-47a6-9b41-9ef4a39c5c01" containerName="main"
Jul 25 01:09:36.149195 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.149176    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"
Jul 25 01:09:36.151287 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.151264    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:09:36.151422 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.151267    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:09:36.158271 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.158237    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"]
Jul 25 01:09:36.252909 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.252868    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7063670e-f033-466c-8023-5958a2a52514-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82216ebf-62t2w\" (UID: \"7063670e-f033-466c-8023-5958a2a52514\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"
Jul 25 01:09:36.353874 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.353835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7063670e-f033-466c-8023-5958a2a52514-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82216ebf-62t2w\" (UID: \"7063670e-f033-466c-8023-5958a2a52514\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"
Jul 25 01:09:36.354572 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.354548    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7063670e-f033-466c-8023-5958a2a52514-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82216ebf-62t2w\" (UID: \"7063670e-f033-466c-8023-5958a2a52514\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"
Jul 25 01:09:36.459939 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.459854    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"
Jul 25 01:09:36.594254 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.594222    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"]
Jul 25 01:09:36.596716 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:09:36.596688    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7063670e_f033_466c_8023_5958a2a52514.slice/crio-05061f095c6e7f84c9b9161cd33631118b7fc454c1c442ba7e0f7cf8c714f425 WatchSource:0}: Error finding container 05061f095c6e7f84c9b9161cd33631118b7fc454c1c442ba7e0f7cf8c714f425: Status 404 returned error can't find the container with id 05061f095c6e7f84c9b9161cd33631118b7fc454c1c442ba7e0f7cf8c714f425
Jul 25 01:09:36.598587 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.598571    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:09:36.856121 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:36.856020    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w" event={"ID":"7063670e-f033-466c-8023-5958a2a52514","Type":"ContainerStarted","Data":"05061f095c6e7f84c9b9161cd33631118b7fc454c1c442ba7e0f7cf8c714f425"}
Jul 25 01:09:39.869470 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:39.869433    2566 generic.go:356] "Generic (PLEG): container finished" podID="7063670e-f033-466c-8023-5958a2a52514" containerID="46b3d582f4c8fde70e5616c2cab5207e06628622683db5b887fccbf5eb30ceb7" exitCode=0
Jul 25 01:09:39.869884 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:39.869498    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w" event={"ID":"7063670e-f033-466c-8023-5958a2a52514","Type":"ContainerDied","Data":"46b3d582f4c8fde70e5616c2cab5207e06628622683db5b887fccbf5eb30ceb7"}
Jul 25 01:09:40.994078 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:40.994030    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"
Jul 25 01:09:41.096500 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:41.096462    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7063670e-f033-466c-8023-5958a2a52514-maas-api-service-ca\") pod \"7063670e-f033-466c-8023-5958a2a52514\" (UID: \"7063670e-f033-466c-8023-5958a2a52514\") "
Jul 25 01:09:41.096815 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:41.096790    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7063670e-f033-466c-8023-5958a2a52514-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7063670e-f033-466c-8023-5958a2a52514" (UID: "7063670e-f033-466c-8023-5958a2a52514"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:09:41.197249 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:41.197163    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7063670e-f033-466c-8023-5958a2a52514-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:09:41.879024 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:41.878993    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"
Jul 25 01:09:41.879202 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:41.878997    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w" event={"ID":"7063670e-f033-466c-8023-5958a2a52514","Type":"ContainerDied","Data":"05061f095c6e7f84c9b9161cd33631118b7fc454c1c442ba7e0f7cf8c714f425"}
Jul 25 01:09:41.879202 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:41.879121    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05061f095c6e7f84c9b9161cd33631118b7fc454c1c442ba7e0f7cf8c714f425"
Jul 25 01:09:49.690430 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:49.690390    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:09:51.221912 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:51.221861    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"]
Jul 25 01:09:51.223241 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:51.223220    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82216ebf-62t2w"]
Jul 25 01:09:52.170787 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:09:52.170752    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7063670e-f033-466c-8023-5958a2a52514" path="/var/lib/kubelet/pods/7063670e-f033-466c-8023-5958a2a52514/volumes"
Jul 25 01:10:01.683114 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:01.683080    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:10:20.457383 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.457342    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"]
Jul 25 01:10:20.457915 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.457697    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7063670e-f033-466c-8023-5958a2a52514" containerName="revoke-keys"
Jul 25 01:10:20.457915 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.457712    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7063670e-f033-466c-8023-5958a2a52514" containerName="revoke-keys"
Jul 25 01:10:20.457915 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.457791    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7063670e-f033-466c-8023-5958a2a52514" containerName="revoke-keys"
Jul 25 01:10:20.464145 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.464117    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"
Jul 25 01:10:20.466398 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.466371    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:10:20.466551 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.466420    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:10:20.467745 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.467716    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"]
Jul 25 01:10:20.584135 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.584104    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a958447e-7969-4e8d-acbd-c922d14f1240-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj\" (UID: \"a958447e-7969-4e8d-acbd-c922d14f1240\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"
Jul 25 01:10:20.685027 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.685000    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a958447e-7969-4e8d-acbd-c922d14f1240-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj\" (UID: \"a958447e-7969-4e8d-acbd-c922d14f1240\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"
Jul 25 01:10:20.685643 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.685601    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a958447e-7969-4e8d-acbd-c922d14f1240-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj\" (UID: \"a958447e-7969-4e8d-acbd-c922d14f1240\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"
Jul 25 01:10:20.777296 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.777235    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"
Jul 25 01:10:20.907168 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:20.907145    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"]
Jul 25 01:10:20.909670 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:10:20.909620    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda958447e_7969_4e8d_acbd_c922d14f1240.slice/crio-f1bc0f075d4e053319207a594ddfd301c1495e39d5c960604b8b9c63f4b87da9 WatchSource:0}: Error finding container f1bc0f075d4e053319207a594ddfd301c1495e39d5c960604b8b9c63f4b87da9: Status 404 returned error can't find the container with id f1bc0f075d4e053319207a594ddfd301c1495e39d5c960604b8b9c63f4b87da9
Jul 25 01:10:21.040850 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:21.040799    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj" event={"ID":"a958447e-7969-4e8d-acbd-c922d14f1240","Type":"ContainerStarted","Data":"f1bc0f075d4e053319207a594ddfd301c1495e39d5c960604b8b9c63f4b87da9"}
Jul 25 01:10:22.046694 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:22.046656    2566 generic.go:356] "Generic (PLEG): container finished" podID="a958447e-7969-4e8d-acbd-c922d14f1240" containerID="34553ecf2725b32c5e1860411f1f2f13c35e02f06c56e9fcf968e45a8b0ce76c" exitCode=0
Jul 25 01:10:22.047127 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:22.046751    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj" event={"ID":"a958447e-7969-4e8d-acbd-c922d14f1240","Type":"ContainerDied","Data":"34553ecf2725b32c5e1860411f1f2f13c35e02f06c56e9fcf968e45a8b0ce76c"}
Jul 25 01:10:23.174620 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:23.174599    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"
Jul 25 01:10:23.303602 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:23.303537    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a958447e-7969-4e8d-acbd-c922d14f1240-maas-api-service-ca\") pod \"a958447e-7969-4e8d-acbd-c922d14f1240\" (UID: \"a958447e-7969-4e8d-acbd-c922d14f1240\") "
Jul 25 01:10:23.303847 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:23.303827    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a958447e-7969-4e8d-acbd-c922d14f1240-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a958447e-7969-4e8d-acbd-c922d14f1240" (UID: "a958447e-7969-4e8d-acbd-c922d14f1240"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:10:23.404928 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:23.404904    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a958447e-7969-4e8d-acbd-c922d14f1240-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:10:24.057104 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:24.057077    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"
Jul 25 01:10:24.057259 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:24.057050    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj" event={"ID":"a958447e-7969-4e8d-acbd-c922d14f1240","Type":"ContainerDied","Data":"f1bc0f075d4e053319207a594ddfd301c1495e39d5c960604b8b9c63f4b87da9"}
Jul 25 01:10:24.057259 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:24.057206    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1bc0f075d4e053319207a594ddfd301c1495e39d5c960604b8b9c63f4b87da9"
Jul 25 01:10:34.084290 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:34.084242    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:10:35.527238 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:35.527198    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"]
Jul 25 01:10:35.529540 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:35.529509    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6fd8e2-w9ctj"]
Jul 25 01:10:36.171527 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:36.171492    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a958447e-7969-4e8d-acbd-c922d14f1240" path="/var/lib/kubelet/pods/a958447e-7969-4e8d-acbd-c922d14f1240/volumes"
Jul 25 01:10:44.994681 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:10:44.994600    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:11:04.988459 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:04.988431    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"]
Jul 25 01:11:04.988966 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:04.988694    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a958447e-7969-4e8d-acbd-c922d14f1240" containerName="revoke-keys"
Jul 25 01:11:04.988966 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:04.988708    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a958447e-7969-4e8d-acbd-c922d14f1240" containerName="revoke-keys"
Jul 25 01:11:04.988966 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:04.988761    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a958447e-7969-4e8d-acbd-c922d14f1240" containerName="revoke-keys"
Jul 25 01:11:04.991615 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:04.991599    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"
Jul 25 01:11:04.993615 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:04.993593    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:11:04.993739 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:04.993715    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:11:04.998499 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:04.998476    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"]
Jul 25 01:11:05.090078 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:05.090040    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f2bc6a-fd43-464c-999c-d562d6009385-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0f061979-mck8q\" (UID: \"19f2bc6a-fd43-464c-999c-d562d6009385\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"
Jul 25 01:11:05.190785 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:05.190760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f2bc6a-fd43-464c-999c-d562d6009385-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0f061979-mck8q\" (UID: \"19f2bc6a-fd43-464c-999c-d562d6009385\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"
Jul 25 01:11:05.191369 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:05.191352    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f2bc6a-fd43-464c-999c-d562d6009385-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0f061979-mck8q\" (UID: \"19f2bc6a-fd43-464c-999c-d562d6009385\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"
Jul 25 01:11:05.302214 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:05.302167    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"
Jul 25 01:11:05.419791 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:05.419768    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"]
Jul 25 01:11:05.422138 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:11:05.422109    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19f2bc6a_fd43_464c_999c_d562d6009385.slice/crio-f89005480ae9155bbd36b5e61b67accb7b9c8f5f9584bf4b04aa02585a6b1fa7 WatchSource:0}: Error finding container f89005480ae9155bbd36b5e61b67accb7b9c8f5f9584bf4b04aa02585a6b1fa7: Status 404 returned error can't find the container with id f89005480ae9155bbd36b5e61b67accb7b9c8f5f9584bf4b04aa02585a6b1fa7
Jul 25 01:11:06.239793 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:06.239763    2566 generic.go:356] "Generic (PLEG): container finished" podID="19f2bc6a-fd43-464c-999c-d562d6009385" containerID="b8438c51ecd520f108a88c96fe69e25b602b3742023ae3230f8321f0b5de0f6e" exitCode=0
Jul 25 01:11:06.240108 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:06.239842    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q" event={"ID":"19f2bc6a-fd43-464c-999c-d562d6009385","Type":"ContainerDied","Data":"b8438c51ecd520f108a88c96fe69e25b602b3742023ae3230f8321f0b5de0f6e"}
Jul 25 01:11:06.240108 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:06.239888    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q" event={"ID":"19f2bc6a-fd43-464c-999c-d562d6009385","Type":"ContainerStarted","Data":"f89005480ae9155bbd36b5e61b67accb7b9c8f5f9584bf4b04aa02585a6b1fa7"}
Jul 25 01:11:07.367500 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:07.367476    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"
Jul 25 01:11:07.406436 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:07.406412    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f2bc6a-fd43-464c-999c-d562d6009385-maas-api-service-ca\") pod \"19f2bc6a-fd43-464c-999c-d562d6009385\" (UID: \"19f2bc6a-fd43-464c-999c-d562d6009385\") "
Jul 25 01:11:07.406735 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:07.406715    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19f2bc6a-fd43-464c-999c-d562d6009385-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "19f2bc6a-fd43-464c-999c-d562d6009385" (UID: "19f2bc6a-fd43-464c-999c-d562d6009385"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:11:07.507622 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:07.507576    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f2bc6a-fd43-464c-999c-d562d6009385-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:11:08.248483 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:08.248455    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q" event={"ID":"19f2bc6a-fd43-464c-999c-d562d6009385","Type":"ContainerDied","Data":"f89005480ae9155bbd36b5e61b67accb7b9c8f5f9584bf4b04aa02585a6b1fa7"}
Jul 25 01:11:08.248483 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:08.248483    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"
Jul 25 01:11:08.248638 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:08.248506    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f89005480ae9155bbd36b5e61b67accb7b9c8f5f9584bf4b04aa02585a6b1fa7"
Jul 25 01:11:18.383875 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:18.383843    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:11:20.063431 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:20.063383    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"]
Jul 25 01:11:20.067547 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:20.067516    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0f061979-mck8q"]
Jul 25 01:11:20.171053 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:20.171019    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19f2bc6a-fd43-464c-999c-d562d6009385" path="/var/lib/kubelet/pods/19f2bc6a-fd43-464c-999c-d562d6009385/volumes"
Jul 25 01:11:30.583635 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:30.583606    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:11:49.650626 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.650591    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"]
Jul 25 01:11:49.651282 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.651003    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f2bc6a-fd43-464c-999c-d562d6009385" containerName="revoke-keys"
Jul 25 01:11:49.651282 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.651024    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f2bc6a-fd43-464c-999c-d562d6009385" containerName="revoke-keys"
Jul 25 01:11:49.651282 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.651111    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="19f2bc6a-fd43-464c-999c-d562d6009385" containerName="revoke-keys"
Jul 25 01:11:49.654118 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.654097    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"
Jul 25 01:11:49.656508 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.656488    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:11:49.656609 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.656537    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:11:49.662138 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.662116    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"]
Jul 25 01:11:49.826000 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.825964    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc022a9e-f0b2-4b2b-8009-e74e397c8f5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s\" (UID: \"dc022a9e-f0b2-4b2b-8009-e74e397c8f5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"
Jul 25 01:11:49.926727 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.926653    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc022a9e-f0b2-4b2b-8009-e74e397c8f5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s\" (UID: \"dc022a9e-f0b2-4b2b-8009-e74e397c8f5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"
Jul 25 01:11:49.927250 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.927230    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc022a9e-f0b2-4b2b-8009-e74e397c8f5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s\" (UID: \"dc022a9e-f0b2-4b2b-8009-e74e397c8f5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"
Jul 25 01:11:49.964688 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:49.964664    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"
Jul 25 01:11:50.087113 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:50.087087    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"]
Jul 25 01:11:50.089352 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:11:50.089317    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc022a9e_f0b2_4b2b_8009_e74e397c8f5b.slice/crio-9f53a6f427966f3caec9ce65ca09fe45a1d0109ef4f632b07d714136546d1a06 WatchSource:0}: Error finding container 9f53a6f427966f3caec9ce65ca09fe45a1d0109ef4f632b07d714136546d1a06: Status 404 returned error can't find the container with id 9f53a6f427966f3caec9ce65ca09fe45a1d0109ef4f632b07d714136546d1a06
Jul 25 01:11:50.426654 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:50.426617    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s" event={"ID":"dc022a9e-f0b2-4b2b-8009-e74e397c8f5b","Type":"ContainerStarted","Data":"f3477c21c58ff2c38966bc07a8f7517c7a74c39448438135b27a9096bb7aab2d"}
Jul 25 01:11:50.426816 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:50.426672    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s" event={"ID":"dc022a9e-f0b2-4b2b-8009-e74e397c8f5b","Type":"ContainerStarted","Data":"9f53a6f427966f3caec9ce65ca09fe45a1d0109ef4f632b07d714136546d1a06"}
Jul 25 01:11:50.442833 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:50.442793    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s" podStartSLOduration=1.442778515 podStartE2EDuration="1.442778515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:11:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:50.441374452 +0000 UTC m=+2132.829896857" watchObservedRunningTime="2026-07-25 01:11:50.442778515 +0000 UTC m=+2132.831300913"
Jul 25 01:11:51.431464 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:51.431429    2566 generic.go:356] "Generic (PLEG): container finished" podID="dc022a9e-f0b2-4b2b-8009-e74e397c8f5b" containerID="f3477c21c58ff2c38966bc07a8f7517c7a74c39448438135b27a9096bb7aab2d" exitCode=0
Jul 25 01:11:51.431836 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:51.431516    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s" event={"ID":"dc022a9e-f0b2-4b2b-8009-e74e397c8f5b","Type":"ContainerDied","Data":"f3477c21c58ff2c38966bc07a8f7517c7a74c39448438135b27a9096bb7aab2d"}
Jul 25 01:11:52.560674 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:52.560649    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"
Jul 25 01:11:52.750555 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:52.750470    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc022a9e-f0b2-4b2b-8009-e74e397c8f5b-maas-api-service-ca\") pod \"dc022a9e-f0b2-4b2b-8009-e74e397c8f5b\" (UID: \"dc022a9e-f0b2-4b2b-8009-e74e397c8f5b\") "
Jul 25 01:11:52.750858 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:52.750835    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc022a9e-f0b2-4b2b-8009-e74e397c8f5b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc022a9e-f0b2-4b2b-8009-e74e397c8f5b" (UID: "dc022a9e-f0b2-4b2b-8009-e74e397c8f5b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:11:52.851625 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:52.851588    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc022a9e-f0b2-4b2b-8009-e74e397c8f5b-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:11:53.440288 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:53.440257    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"
Jul 25 01:11:53.440458 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:53.440292    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s" event={"ID":"dc022a9e-f0b2-4b2b-8009-e74e397c8f5b","Type":"ContainerDied","Data":"9f53a6f427966f3caec9ce65ca09fe45a1d0109ef4f632b07d714136546d1a06"}
Jul 25 01:11:53.440458 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:11:53.440340    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f53a6f427966f3caec9ce65ca09fe45a1d0109ef4f632b07d714136546d1a06"
Jul 25 01:12:03.189208 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:12:03.189167    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:12:04.720883 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:12:04.720846    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"]
Jul 25 01:12:04.723537 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:12:04.723512    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a271f0bb-gxw8s"]
Jul 25 01:12:06.170941 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:12:06.170907    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc022a9e-f0b2-4b2b-8009-e74e397c8f5b" path="/var/lib/kubelet/pods/dc022a9e-f0b2-4b2b-8009-e74e397c8f5b/volumes"
Jul 25 01:13:29.997854 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:29.997821    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:13:40.895076 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:40.894971    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:13:45.863420 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.863380    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"]
Jul 25 01:13:45.863810 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.863790    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc022a9e-f0b2-4b2b-8009-e74e397c8f5b" containerName="revoke-keys"
Jul 25 01:13:45.863866 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.863815    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc022a9e-f0b2-4b2b-8009-e74e397c8f5b" containerName="revoke-keys"
Jul 25 01:13:45.863932 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.863919    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc022a9e-f0b2-4b2b-8009-e74e397c8f5b" containerName="revoke-keys"
Jul 25 01:13:45.867196 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.867178    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:13:45.869291 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.869271    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-818dfc55-dockercfg-7cpsk\""
Jul 25 01:13:45.869390 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.869273    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-818dfc55\""
Jul 25 01:13:45.876807 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.876781    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"]
Jul 25 01:13:45.888355 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.888327    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce0daace-7264-45c7-87b2-8cb76da5412b-plugins-config-volume\") pod \"payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn\" (UID: \"ce0daace-7264-45c7-87b2-8cb76da5412b\") " pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:13:45.888443 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.888409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k67p6\" (UniqueName: \"kubernetes.io/projected/ce0daace-7264-45c7-87b2-8cb76da5412b-kube-api-access-k67p6\") pod \"payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn\" (UID: \"ce0daace-7264-45c7-87b2-8cb76da5412b\") " pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:13:45.989312 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.989278    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k67p6\" (UniqueName: \"kubernetes.io/projected/ce0daace-7264-45c7-87b2-8cb76da5412b-kube-api-access-k67p6\") pod \"payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn\" (UID: \"ce0daace-7264-45c7-87b2-8cb76da5412b\") " pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:13:45.989439 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.989341    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce0daace-7264-45c7-87b2-8cb76da5412b-plugins-config-volume\") pod \"payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn\" (UID: \"ce0daace-7264-45c7-87b2-8cb76da5412b\") " pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:13:45.990045 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:45.990027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce0daace-7264-45c7-87b2-8cb76da5412b-plugins-config-volume\") pod \"payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn\" (UID: \"ce0daace-7264-45c7-87b2-8cb76da5412b\") " pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:13:46.000396 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:46.000375    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k67p6\" (UniqueName: \"kubernetes.io/projected/ce0daace-7264-45c7-87b2-8cb76da5412b-kube-api-access-k67p6\") pod \"payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn\" (UID: \"ce0daace-7264-45c7-87b2-8cb76da5412b\") " pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:13:46.177662 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:46.177634    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:13:46.307201 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:46.307168    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"]
Jul 25 01:13:46.309685 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:13:46.309660    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0daace_7264_45c7_87b2_8cb76da5412b.slice/crio-66ab4fe82b839bda168e0658d2561640d8212680839e784ac1c6045a4967ccd6 WatchSource:0}: Error finding container 66ab4fe82b839bda168e0658d2561640d8212680839e784ac1c6045a4967ccd6: Status 404 returned error can't find the container with id 66ab4fe82b839bda168e0658d2561640d8212680839e784ac1c6045a4967ccd6
Jul 25 01:13:46.938710 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:46.938671    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn" event={"ID":"ce0daace-7264-45c7-87b2-8cb76da5412b","Type":"ContainerStarted","Data":"66ab4fe82b839bda168e0658d2561640d8212680839e784ac1c6045a4967ccd6"}
Jul 25 01:13:47.950750 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:47.950714    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn" event={"ID":"ce0daace-7264-45c7-87b2-8cb76da5412b","Type":"ContainerStarted","Data":"61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8"}
Jul 25 01:13:47.951108 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:47.950799    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:13:47.968331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:47.968287    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn" podStartSLOduration=1.454969942 podStartE2EDuration="2.968274003s" totalImagesPullingTime="1.513304061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:47.966113043 +0000 UTC m=+2250.354635449" watchObservedRunningTime="2026-07-25 01:13:47.968274003 +0000 UTC m=+2250.356796405"
Jul 25 01:13:49.990672 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:49.990637    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:13:56.574470 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:56.574432    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"]
Jul 25 01:13:56.577455 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:56.577438    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"
Jul 25 01:13:56.579771 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:56.579748    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:13:56.579899 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:56.579753    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:13:56.585436 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:56.585410    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"]
Jul 25 01:13:56.663226 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:56.663182    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d747e28a-41c6-4450-bc28-e5d7b30934fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp\" (UID: \"d747e28a-41c6-4450-bc28-e5d7b30934fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"
Jul 25 01:13:56.764511 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:56.764477    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d747e28a-41c6-4450-bc28-e5d7b30934fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp\" (UID: \"d747e28a-41c6-4450-bc28-e5d7b30934fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"
Jul 25 01:13:56.765149 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:56.765131    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d747e28a-41c6-4450-bc28-e5d7b30934fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp\" (UID: \"d747e28a-41c6-4450-bc28-e5d7b30934fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"
Jul 25 01:13:56.888164 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:56.888136    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"
Jul 25 01:13:57.009024 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:57.009007    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"]
Jul 25 01:13:57.011565 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:13:57.011538    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd747e28a_41c6_4450_bc28_e5d7b30934fd.slice/crio-806bc4aaf68ec76d5dcc662af10fd5b77f802c512a36d5908fa65193dccd267b WatchSource:0}: Error finding container 806bc4aaf68ec76d5dcc662af10fd5b77f802c512a36d5908fa65193dccd267b: Status 404 returned error can't find the container with id 806bc4aaf68ec76d5dcc662af10fd5b77f802c512a36d5908fa65193dccd267b
Jul 25 01:13:57.999251 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:57.999177    2566 generic.go:356] "Generic (PLEG): container finished" podID="d747e28a-41c6-4450-bc28-e5d7b30934fd" containerID="111e86f0dc7ec0fa1a6dfae1df3eb279f15566e02c717aed5be652870b78a258" exitCode=0
Jul 25 01:13:57.999251 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:57.999225    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp" event={"ID":"d747e28a-41c6-4450-bc28-e5d7b30934fd","Type":"ContainerDied","Data":"111e86f0dc7ec0fa1a6dfae1df3eb279f15566e02c717aed5be652870b78a258"}
Jul 25 01:13:57.999623 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:57.999269    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp" event={"ID":"d747e28a-41c6-4450-bc28-e5d7b30934fd","Type":"ContainerStarted","Data":"806bc4aaf68ec76d5dcc662af10fd5b77f802c512a36d5908fa65193dccd267b"}
Jul 25 01:13:58.957350 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:58.957323    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:13:59.131447 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:59.131422    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"
Jul 25 01:13:59.180655 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:59.180631    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d747e28a-41c6-4450-bc28-e5d7b30934fd-maas-api-service-ca\") pod \"d747e28a-41c6-4450-bc28-e5d7b30934fd\" (UID: \"d747e28a-41c6-4450-bc28-e5d7b30934fd\") "
Jul 25 01:13:59.180966 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:59.180944    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d747e28a-41c6-4450-bc28-e5d7b30934fd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d747e28a-41c6-4450-bc28-e5d7b30934fd" (UID: "d747e28a-41c6-4450-bc28-e5d7b30934fd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:13:59.281610 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:13:59.281543    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d747e28a-41c6-4450-bc28-e5d7b30934fd-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:14:00.010154 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:00.010115    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp" event={"ID":"d747e28a-41c6-4450-bc28-e5d7b30934fd","Type":"ContainerDied","Data":"806bc4aaf68ec76d5dcc662af10fd5b77f802c512a36d5908fa65193dccd267b"}
Jul 25 01:14:00.010154 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:00.010142    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"
Jul 25 01:14:00.010374 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:00.010172    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="806bc4aaf68ec76d5dcc662af10fd5b77f802c512a36d5908fa65193dccd267b"
Jul 25 01:14:06.975141 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:06.975101    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"]
Jul 25 01:14:06.975689 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:06.975353    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn" podUID="ce0daace-7264-45c7-87b2-8cb76da5412b" containerName="payload-processing" containerID="cri-o://61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8" gracePeriod=30
Jul 25 01:14:07.221775 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:07.221753    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:14:07.339607 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:07.339544    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce0daace-7264-45c7-87b2-8cb76da5412b-plugins-config-volume\") pod \"ce0daace-7264-45c7-87b2-8cb76da5412b\" (UID: \"ce0daace-7264-45c7-87b2-8cb76da5412b\") "
Jul 25 01:14:07.339607 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:07.339581    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k67p6\" (UniqueName: \"kubernetes.io/projected/ce0daace-7264-45c7-87b2-8cb76da5412b-kube-api-access-k67p6\") pod \"ce0daace-7264-45c7-87b2-8cb76da5412b\" (UID: \"ce0daace-7264-45c7-87b2-8cb76da5412b\") "
Jul 25 01:14:07.339927 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:07.339903    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce0daace-7264-45c7-87b2-8cb76da5412b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce0daace-7264-45c7-87b2-8cb76da5412b" (UID: "ce0daace-7264-45c7-87b2-8cb76da5412b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:07.341588 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:07.341560    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce0daace-7264-45c7-87b2-8cb76da5412b-kube-api-access-k67p6" (OuterVolumeSpecName: "kube-api-access-k67p6") pod "ce0daace-7264-45c7-87b2-8cb76da5412b" (UID: "ce0daace-7264-45c7-87b2-8cb76da5412b"). InnerVolumeSpecName "kube-api-access-k67p6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:14:07.440128 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:07.440101    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce0daace-7264-45c7-87b2-8cb76da5412b-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:14:07.440128 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:07.440125    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k67p6\" (UniqueName: \"kubernetes.io/projected/ce0daace-7264-45c7-87b2-8cb76da5412b-kube-api-access-k67p6\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:14:08.047372 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:08.047330    2566 generic.go:356] "Generic (PLEG): container finished" podID="ce0daace-7264-45c7-87b2-8cb76da5412b" containerID="61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8" exitCode=0
Jul 25 01:14:08.047743 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:08.047383    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"
Jul 25 01:14:08.047743 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:08.047424    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn" event={"ID":"ce0daace-7264-45c7-87b2-8cb76da5412b","Type":"ContainerDied","Data":"61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8"}
Jul 25 01:14:08.047743 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:08.047488    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn" event={"ID":"ce0daace-7264-45c7-87b2-8cb76da5412b","Type":"ContainerDied","Data":"66ab4fe82b839bda168e0658d2561640d8212680839e784ac1c6045a4967ccd6"}
Jul 25 01:14:08.047743 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:08.047514    2566 scope.go:117] "RemoveContainer" containerID="61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8"
Jul 25 01:14:08.056973 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:08.056953    2566 scope.go:117] "RemoveContainer" containerID="61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8"
Jul 25 01:14:08.057293 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:14:08.057273    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8\": container with ID starting with 61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8 not found: ID does not exist" containerID="61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8"
Jul 25 01:14:08.057342 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:08.057301    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8"} err="failed to get container status \"61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8\": rpc error: code = NotFound desc = could not find container \"61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8\": container with ID starting with 61ba9c771b33ec710b9be92a65642e7dbc83d410564d9adcd9dcad4de40044b8 not found: ID does not exist"
Jul 25 01:14:08.067696 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:08.067664    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"]
Jul 25 01:14:08.071604 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:08.071579    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-818dfc55-55885d6646-zb2mn"]
Jul 25 01:14:08.170770 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:08.170746    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce0daace-7264-45c7-87b2-8cb76da5412b" path="/var/lib/kubelet/pods/ce0daace-7264-45c7-87b2-8cb76da5412b/volumes"
Jul 25 01:14:09.693380 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:09.693339    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:14:11.648497 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:11.648456    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"]
Jul 25 01:14:11.650581 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:11.650555    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-818dfc55-dxgqp"]
Jul 25 01:14:12.171018 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:12.170989    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d747e28a-41c6-4450-bc28-e5d7b30934fd" path="/var/lib/kubelet/pods/d747e28a-41c6-4450-bc28-e5d7b30934fd/volumes"
Jul 25 01:14:42.037594 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.037553    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"]
Jul 25 01:14:42.038042 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.037837    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce0daace-7264-45c7-87b2-8cb76da5412b" containerName="payload-processing"
Jul 25 01:14:42.038042 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.037849    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce0daace-7264-45c7-87b2-8cb76da5412b" containerName="payload-processing"
Jul 25 01:14:42.038042 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.037867    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d747e28a-41c6-4450-bc28-e5d7b30934fd" containerName="revoke-keys"
Jul 25 01:14:42.038042 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.037873    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d747e28a-41c6-4450-bc28-e5d7b30934fd" containerName="revoke-keys"
Jul 25 01:14:42.038042 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.037917    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce0daace-7264-45c7-87b2-8cb76da5412b" containerName="payload-processing"
Jul 25 01:14:42.038042 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.037925    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d747e28a-41c6-4450-bc28-e5d7b30934fd" containerName="revoke-keys"
Jul 25 01:14:42.040703 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.040686    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:14:42.043409 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.043381    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-98d19c4d-dockercfg-7tvlt\""
Jul 25 01:14:42.043409 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.043392    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-98d19c4d\""
Jul 25 01:14:42.068001 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.067983    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"]
Jul 25 01:14:42.069593 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.069565    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"]
Jul 25 01:14:42.072739 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.072723    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:14:42.087738 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.087714    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"]
Jul 25 01:14:42.187029 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.187004    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4mf6\" (UniqueName: \"kubernetes.io/projected/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-kube-api-access-z4mf6\") pod \"payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n\" (UID: \"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:14:42.187189 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.187033    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n\" (UID: \"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:14:42.187189 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.187056    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4hmv\" (UniqueName: \"kubernetes.io/projected/a29ce51e-7d81-4a5b-92bd-e98110e6954b-kube-api-access-l4hmv\") pod \"payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz\" (UID: \"a29ce51e-7d81-4a5b-92bd-e98110e6954b\") " pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:14:42.187189 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.187165    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a29ce51e-7d81-4a5b-92bd-e98110e6954b-plugins-config-volume\") pod \"payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz\" (UID: \"a29ce51e-7d81-4a5b-92bd-e98110e6954b\") " pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:14:42.288398 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.288338    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4mf6\" (UniqueName: \"kubernetes.io/projected/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-kube-api-access-z4mf6\") pod \"payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n\" (UID: \"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:14:42.288398 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.288371    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n\" (UID: \"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:14:42.288398 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.288395    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4hmv\" (UniqueName: \"kubernetes.io/projected/a29ce51e-7d81-4a5b-92bd-e98110e6954b-kube-api-access-l4hmv\") pod \"payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz\" (UID: \"a29ce51e-7d81-4a5b-92bd-e98110e6954b\") " pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:14:42.288569 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.288421    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a29ce51e-7d81-4a5b-92bd-e98110e6954b-plugins-config-volume\") pod \"payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz\" (UID: \"a29ce51e-7d81-4a5b-92bd-e98110e6954b\") " pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:14:42.289115 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.289091    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a29ce51e-7d81-4a5b-92bd-e98110e6954b-plugins-config-volume\") pod \"payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz\" (UID: \"a29ce51e-7d81-4a5b-92bd-e98110e6954b\") " pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:14:42.289115 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.289111    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n\" (UID: \"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:14:42.296077 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.296048    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4hmv\" (UniqueName: \"kubernetes.io/projected/a29ce51e-7d81-4a5b-92bd-e98110e6954b-kube-api-access-l4hmv\") pod \"payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz\" (UID: \"a29ce51e-7d81-4a5b-92bd-e98110e6954b\") " pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:14:42.296163 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.296121    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4mf6\" (UniqueName: \"kubernetes.io/projected/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-kube-api-access-z4mf6\") pod \"payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n\" (UID: \"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:14:42.350875 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.350852    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:14:42.381728 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.381701    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:14:42.484354 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.484265    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"]
Jul 25 01:14:42.487223 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:14:42.487189    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc3c58b7_03cc_4c47_b6b9_cdfaf1a2ee2c.slice/crio-08c63a038768d95cb038332160c21a8a1fdeb9df9abd543adcfdcf1e83023a17 WatchSource:0}: Error finding container 08c63a038768d95cb038332160c21a8a1fdeb9df9abd543adcfdcf1e83023a17: Status 404 returned error can't find the container with id 08c63a038768d95cb038332160c21a8a1fdeb9df9abd543adcfdcf1e83023a17
Jul 25 01:14:42.519152 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:42.519125    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"]
Jul 25 01:14:42.521959 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:14:42.521934    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29ce51e_7d81_4a5b_92bd_e98110e6954b.slice/crio-24b18d68c13571811a5943e6f3981ece4deb204bac7df9eb44c1d29fb43cd48c WatchSource:0}: Error finding container 24b18d68c13571811a5943e6f3981ece4deb204bac7df9eb44c1d29fb43cd48c: Status 404 returned error can't find the container with id 24b18d68c13571811a5943e6f3981ece4deb204bac7df9eb44c1d29fb43cd48c
Jul 25 01:14:43.208574 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:43.208537    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz" event={"ID":"a29ce51e-7d81-4a5b-92bd-e98110e6954b","Type":"ContainerStarted","Data":"9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b"}
Jul 25 01:14:43.208962 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:43.208696    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:14:43.208962 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:43.208751    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz" event={"ID":"a29ce51e-7d81-4a5b-92bd-e98110e6954b","Type":"ContainerStarted","Data":"24b18d68c13571811a5943e6f3981ece4deb204bac7df9eb44c1d29fb43cd48c"}
Jul 25 01:14:43.210018 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:43.209995    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n" event={"ID":"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c","Type":"ContainerStarted","Data":"a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d"}
Jul 25 01:14:43.210144 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:43.210031    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n" event={"ID":"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c","Type":"ContainerStarted","Data":"08c63a038768d95cb038332160c21a8a1fdeb9df9abd543adcfdcf1e83023a17"}
Jul 25 01:14:43.210144 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:43.210086    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:14:43.225909 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:43.225861    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz" podStartSLOduration=1.225845508 podStartE2EDuration="1.225845508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:14:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:14:43.22345981 +0000 UTC m=+2305.611982212" watchObservedRunningTime="2026-07-25 01:14:43.225845508 +0000 UTC m=+2305.614367914"
Jul 25 01:14:43.239555 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:43.239518    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n" podStartSLOduration=1.23950443 podStartE2EDuration="1.23950443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:14:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:14:43.238631135 +0000 UTC m=+2305.627153540" watchObservedRunningTime="2026-07-25 01:14:43.23950443 +0000 UTC m=+2305.628026863"
Jul 25 01:14:48.091786 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:48.091745    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:14:54.215337 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:54.215301    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:14:54.215803 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:14:54.215374    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:15:00.129134 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:00.129093    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c"]
Jul 25 01:15:00.132577 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:00.132558    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c"
Jul 25 01:15:00.134530 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:00.134506    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:15:00.138970 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:00.138939    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c"]
Jul 25 01:15:00.144965 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:00.144935    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c"
Jul 25 01:15:00.476775 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:00.476622    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c"]
Jul 25 01:15:00.479131 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:15:00.479104    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ba3ea9_061e_4fa7_a61b_538086fc1b5b.slice/crio-1648effb2f618e06b1fc138afe5c772bde00d4782b7f866fb342864106750cd7 WatchSource:0}: Error finding container 1648effb2f618e06b1fc138afe5c772bde00d4782b7f866fb342864106750cd7: Status 404 returned error can't find the container with id 1648effb2f618e06b1fc138afe5c772bde00d4782b7f866fb342864106750cd7
Jul 25 01:15:01.294577 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:01.294552    2566 generic.go:356] "Generic (PLEG): container finished" podID="58ba3ea9-061e-4fa7-a61b-538086fc1b5b" containerID="c5a3a15a2d1ce0324e032b4326be20a21e1d8855f3cb39a57321cc6a77a4478a" exitCode=0
Jul 25 01:15:01.294871 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:01.294637    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c" event={"ID":"58ba3ea9-061e-4fa7-a61b-538086fc1b5b","Type":"ContainerDied","Data":"c5a3a15a2d1ce0324e032b4326be20a21e1d8855f3cb39a57321cc6a77a4478a"}
Jul 25 01:15:01.294871 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:01.294686    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c" event={"ID":"58ba3ea9-061e-4fa7-a61b-538086fc1b5b","Type":"ContainerStarted","Data":"1648effb2f618e06b1fc138afe5c772bde00d4782b7f866fb342864106750cd7"}
Jul 25 01:15:02.428557 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:02.428534    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c"
Jul 25 01:15:03.304196 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:03.304165    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c"
Jul 25 01:15:03.304392 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:03.304192    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c" event={"ID":"58ba3ea9-061e-4fa7-a61b-538086fc1b5b","Type":"ContainerDied","Data":"1648effb2f618e06b1fc138afe5c772bde00d4782b7f866fb342864106750cd7"}
Jul 25 01:15:03.304392 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:03.304237    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1648effb2f618e06b1fc138afe5c772bde00d4782b7f866fb342864106750cd7"
Jul 25 01:15:11.094037 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:11.093932    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:15:30.896458 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:30.896417    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:15:35.704052 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.704011    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"]
Jul 25 01:15:35.704476 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.704364    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58ba3ea9-061e-4fa7-a61b-538086fc1b5b" containerName="cleanup"
Jul 25 01:15:35.704476 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.704379    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="58ba3ea9-061e-4fa7-a61b-538086fc1b5b" containerName="cleanup"
Jul 25 01:15:35.704476 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.704447    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="58ba3ea9-061e-4fa7-a61b-538086fc1b5b" containerName="cleanup"
Jul 25 01:15:35.708177 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.708142    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"
Jul 25 01:15:35.711298 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.711008    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:15:35.711298 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.711216    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:15:35.719042 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.719010    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"]
Jul 25 01:15:35.761380 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.761353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f12be3c-8b0e-4983-be70-3b2587cd3e73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r\" (UID: \"5f12be3c-8b0e-4983-be70-3b2587cd3e73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"
Jul 25 01:15:35.862641 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.862611    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f12be3c-8b0e-4983-be70-3b2587cd3e73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r\" (UID: \"5f12be3c-8b0e-4983-be70-3b2587cd3e73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"
Jul 25 01:15:35.863214 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:35.863193    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f12be3c-8b0e-4983-be70-3b2587cd3e73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r\" (UID: \"5f12be3c-8b0e-4983-be70-3b2587cd3e73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"
Jul 25 01:15:36.022891 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:36.022829    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"
Jul 25 01:15:36.151516 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:36.151493    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"]
Jul 25 01:15:36.153998 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:15:36.153972    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f12be3c_8b0e_4983_be70_3b2587cd3e73.slice/crio-b58e911718c1d67033de3b57c2b455fb2b162f50d3efb0907d9fc078916d9324 WatchSource:0}: Error finding container b58e911718c1d67033de3b57c2b455fb2b162f50d3efb0907d9fc078916d9324: Status 404 returned error can't find the container with id b58e911718c1d67033de3b57c2b455fb2b162f50d3efb0907d9fc078916d9324
Jul 25 01:15:36.458955 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:36.458929    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r" event={"ID":"5f12be3c-8b0e-4983-be70-3b2587cd3e73","Type":"ContainerStarted","Data":"1f054965cd2dcf012a045576e7998ea830e5beed022de32c7cbece9cc83fc144"}
Jul 25 01:15:36.459058 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:36.458974    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r" event={"ID":"5f12be3c-8b0e-4983-be70-3b2587cd3e73","Type":"ContainerStarted","Data":"b58e911718c1d67033de3b57c2b455fb2b162f50d3efb0907d9fc078916d9324"}
Jul 25 01:15:36.473911 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:36.473862    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r" podStartSLOduration=1.473847265 podStartE2EDuration="1.473847265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:15:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:15:36.472678666 +0000 UTC m=+2358.861201068" watchObservedRunningTime="2026-07-25 01:15:36.473847265 +0000 UTC m=+2358.862369667"
Jul 25 01:15:37.465839 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:37.465801    2566 generic.go:356] "Generic (PLEG): container finished" podID="5f12be3c-8b0e-4983-be70-3b2587cd3e73" containerID="1f054965cd2dcf012a045576e7998ea830e5beed022de32c7cbece9cc83fc144" exitCode=0
Jul 25 01:15:37.466204 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:37.465893    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r" event={"ID":"5f12be3c-8b0e-4983-be70-3b2587cd3e73","Type":"ContainerDied","Data":"1f054965cd2dcf012a045576e7998ea830e5beed022de32c7cbece9cc83fc144"}
Jul 25 01:15:38.597187 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:38.597165    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"
Jul 25 01:15:38.679806 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:38.679777    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f12be3c-8b0e-4983-be70-3b2587cd3e73-maas-api-service-ca\") pod \"5f12be3c-8b0e-4983-be70-3b2587cd3e73\" (UID: \"5f12be3c-8b0e-4983-be70-3b2587cd3e73\") "
Jul 25 01:15:38.680145 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:38.680122    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f12be3c-8b0e-4983-be70-3b2587cd3e73-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f12be3c-8b0e-4983-be70-3b2587cd3e73" (UID: "5f12be3c-8b0e-4983-be70-3b2587cd3e73"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:15:38.780407 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:38.780351    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f12be3c-8b0e-4983-be70-3b2587cd3e73-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:15:39.477170 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:39.477132    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r" event={"ID":"5f12be3c-8b0e-4983-be70-3b2587cd3e73","Type":"ContainerDied","Data":"b58e911718c1d67033de3b57c2b455fb2b162f50d3efb0907d9fc078916d9324"}
Jul 25 01:15:39.477170 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:39.477170    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"
Jul 25 01:15:39.477339 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:39.477193    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b58e911718c1d67033de3b57c2b455fb2b162f50d3efb0907d9fc078916d9324"
Jul 25 01:15:49.992851 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:49.992807    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:15:51.213325 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.213293    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c"]
Jul 25 01:15:51.214869 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.214844    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-98d19c4d-29749035-tkf4c"]
Jul 25 01:15:51.239104 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.239059    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"]
Jul 25 01:15:51.239328 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.239303    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz" podUID="a29ce51e-7d81-4a5b-92bd-e98110e6954b" containerName="payload-processing" containerID="cri-o://9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b" gracePeriod=30
Jul 25 01:15:51.246778 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.246752    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"]
Jul 25 01:15:51.247175 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.247145    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n" podUID="dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c" containerName="payload-pre-processing" containerID="cri-o://a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d" gracePeriod=30
Jul 25 01:15:51.509048 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.509026    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:15:51.511302 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.511285    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:15:51.536053 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.536021    2566 generic.go:356] "Generic (PLEG): container finished" podID="dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c" containerID="a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d" exitCode=0
Jul 25 01:15:51.536173 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.536056    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"
Jul 25 01:15:51.536173 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.536103    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n" event={"ID":"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c","Type":"ContainerDied","Data":"a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d"}
Jul 25 01:15:51.536173 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.536149    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n" event={"ID":"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c","Type":"ContainerDied","Data":"08c63a038768d95cb038332160c21a8a1fdeb9df9abd543adcfdcf1e83023a17"}
Jul 25 01:15:51.536173 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.536170    2566 scope.go:117] "RemoveContainer" containerID="a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d"
Jul 25 01:15:51.537407 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.537386    2566 generic.go:356] "Generic (PLEG): container finished" podID="a29ce51e-7d81-4a5b-92bd-e98110e6954b" containerID="9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b" exitCode=0
Jul 25 01:15:51.537504 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.537416    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz" event={"ID":"a29ce51e-7d81-4a5b-92bd-e98110e6954b","Type":"ContainerDied","Data":"9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b"}
Jul 25 01:15:51.537504 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.537434    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"
Jul 25 01:15:51.537504 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.537464    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz" event={"ID":"a29ce51e-7d81-4a5b-92bd-e98110e6954b","Type":"ContainerDied","Data":"24b18d68c13571811a5943e6f3981ece4deb204bac7df9eb44c1d29fb43cd48c"}
Jul 25 01:15:51.546186 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.546165    2566 scope.go:117] "RemoveContainer" containerID="a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d"
Jul 25 01:15:51.546483 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:15:51.546461    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d\": container with ID starting with a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d not found: ID does not exist" containerID="a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d"
Jul 25 01:15:51.546579 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.546502    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d"} err="failed to get container status \"a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d\": rpc error: code = NotFound desc = could not find container \"a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d\": container with ID starting with a4a5df21cc1024d9cba9a983fd6fea79ae00fcbd463ac00f4463accc364f517d not found: ID does not exist"
Jul 25 01:15:51.546579 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.546525    2566 scope.go:117] "RemoveContainer" containerID="9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b"
Jul 25 01:15:51.553854 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.553836    2566 scope.go:117] "RemoveContainer" containerID="9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b"
Jul 25 01:15:51.554140 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:15:51.554123    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b\": container with ID starting with 9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b not found: ID does not exist" containerID="9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b"
Jul 25 01:15:51.554211 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.554148    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b"} err="failed to get container status \"9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b\": rpc error: code = NotFound desc = could not find container \"9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b\": container with ID starting with 9859c36a27d4230eb95caa63f6df00e85b6b7ab53f9f0b7b7e5af6b10e18558b not found: ID does not exist"
Jul 25 01:15:51.569522 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.569504    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-plugins-config-volume\") pod \"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c\" (UID: \"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c\") "
Jul 25 01:15:51.569623 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.569544    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4hmv\" (UniqueName: \"kubernetes.io/projected/a29ce51e-7d81-4a5b-92bd-e98110e6954b-kube-api-access-l4hmv\") pod \"a29ce51e-7d81-4a5b-92bd-e98110e6954b\" (UID: \"a29ce51e-7d81-4a5b-92bd-e98110e6954b\") "
Jul 25 01:15:51.569623 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.569577    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a29ce51e-7d81-4a5b-92bd-e98110e6954b-plugins-config-volume\") pod \"a29ce51e-7d81-4a5b-92bd-e98110e6954b\" (UID: \"a29ce51e-7d81-4a5b-92bd-e98110e6954b\") "
Jul 25 01:15:51.569734 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.569694    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4mf6\" (UniqueName: \"kubernetes.io/projected/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-kube-api-access-z4mf6\") pod \"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c\" (UID: \"dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c\") "
Jul 25 01:15:51.569900 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.569876    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c" (UID: "dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:15:51.569955 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.569904    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a29ce51e-7d81-4a5b-92bd-e98110e6954b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a29ce51e-7d81-4a5b-92bd-e98110e6954b" (UID: "a29ce51e-7d81-4a5b-92bd-e98110e6954b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:15:51.571400 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.571382    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a29ce51e-7d81-4a5b-92bd-e98110e6954b-kube-api-access-l4hmv" (OuterVolumeSpecName: "kube-api-access-l4hmv") pod "a29ce51e-7d81-4a5b-92bd-e98110e6954b" (UID: "a29ce51e-7d81-4a5b-92bd-e98110e6954b"). InnerVolumeSpecName "kube-api-access-l4hmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:15:51.571490 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.571425    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-kube-api-access-z4mf6" (OuterVolumeSpecName: "kube-api-access-z4mf6") pod "dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c" (UID: "dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c"). InnerVolumeSpecName "kube-api-access-z4mf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:15:51.670506 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.670483    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:15:51.670605 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.670507    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4hmv\" (UniqueName: \"kubernetes.io/projected/a29ce51e-7d81-4a5b-92bd-e98110e6954b-kube-api-access-l4hmv\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:15:51.670605 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.670520    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a29ce51e-7d81-4a5b-92bd-e98110e6954b-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:15:51.670605 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.670529    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4mf6\" (UniqueName: \"kubernetes.io/projected/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c-kube-api-access-z4mf6\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:15:51.860249 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.860226    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"]
Jul 25 01:15:51.864964 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.864942    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-98d19c4d-6bbccd858b-lk7nz"]
Jul 25 01:15:51.873419 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.873400    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"]
Jul 25 01:15:51.876856 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:51.876837    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-98d19c4d-85bfb4d9b4-d8c6n"]
Jul 25 01:15:52.171124 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:52.171100    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58ba3ea9-061e-4fa7-a61b-538086fc1b5b" path="/var/lib/kubelet/pods/58ba3ea9-061e-4fa7-a61b-538086fc1b5b/volumes"
Jul 25 01:15:52.171409 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:52.171396    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a29ce51e-7d81-4a5b-92bd-e98110e6954b" path="/var/lib/kubelet/pods/a29ce51e-7d81-4a5b-92bd-e98110e6954b/volumes"
Jul 25 01:15:52.171697 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:52.171685    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c" path="/var/lib/kubelet/pods/dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c/volumes"
Jul 25 01:15:55.778858 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:55.778821    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"]
Jul 25 01:15:55.780173 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:55.780149    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98d19c4d-2sp8r"]
Jul 25 01:15:56.171191 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:15:56.171157    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f12be3c-8b0e-4983-be70-3b2587cd3e73" path="/var/lib/kubelet/pods/5f12be3c-8b0e-4983-be70-3b2587cd3e73/volumes"
Jul 25 01:16:00.395872 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:16:00.395822    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:16:18.202607 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:16:18.202577    2566 scope.go:117] "RemoveContainer" containerID="46b3d582f4c8fde70e5616c2cab5207e06628622683db5b887fccbf5eb30ceb7"
Jul 25 01:16:51.094320 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:16:51.094236    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:17:03.986787 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:03.986744    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:17:18.229139 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:18.229104    2566 scope.go:117] "RemoveContainer" containerID="b8438c51ecd520f108a88c96fe69e25b602b3742023ae3230f8321f0b5de0f6e"
Jul 25 01:17:18.237404 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:18.237379    2566 scope.go:117] "RemoveContainer" containerID="34553ecf2725b32c5e1860411f1f2f13c35e02f06c56e9fcf968e45a8b0ce76c"
Jul 25 01:17:46.361045 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.361011    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"]
Jul 25 01:17:46.361616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.361463    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f12be3c-8b0e-4983-be70-3b2587cd3e73" containerName="revoke-keys"
Jul 25 01:17:46.361616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.361482    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f12be3c-8b0e-4983-be70-3b2587cd3e73" containerName="revoke-keys"
Jul 25 01:17:46.361616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.361495    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c" containerName="payload-pre-processing"
Jul 25 01:17:46.361616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.361504    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c" containerName="payload-pre-processing"
Jul 25 01:17:46.361616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.361528    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a29ce51e-7d81-4a5b-92bd-e98110e6954b" containerName="payload-processing"
Jul 25 01:17:46.361616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.361536    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a29ce51e-7d81-4a5b-92bd-e98110e6954b" containerName="payload-processing"
Jul 25 01:17:46.361616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.361612    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f12be3c-8b0e-4983-be70-3b2587cd3e73" containerName="revoke-keys"
Jul 25 01:17:46.361961 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.361622    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a29ce51e-7d81-4a5b-92bd-e98110e6954b" containerName="payload-processing"
Jul 25 01:17:46.361961 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.361635    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc3c58b7-03cc-4c47-b6b9-cdfaf1a2ee2c" containerName="payload-pre-processing"
Jul 25 01:17:46.364637 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.364616    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:17:46.372310 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.372289    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-d08be1-dockercfg-dgshz\""
Jul 25 01:17:46.372414 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.372375    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-d08be1\""
Jul 25 01:17:46.383845 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.383817    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"]
Jul 25 01:17:46.395392 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.395368    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b1796bb-f4fd-491e-b48e-0bdbd6447870-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds\" (UID: \"1b1796bb-f4fd-491e-b48e-0bdbd6447870\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:17:46.395496 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.395442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgr47\" (UniqueName: \"kubernetes.io/projected/1b1796bb-f4fd-491e-b48e-0bdbd6447870-kube-api-access-hgr47\") pod \"payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds\" (UID: \"1b1796bb-f4fd-491e-b48e-0bdbd6447870\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:17:46.403991 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.403969    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"]
Jul 25 01:17:46.407023 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.407008    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:17:46.416843 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.416824    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"]
Jul 25 01:17:46.496376 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.496346    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl4d8\" (UniqueName: \"kubernetes.io/projected/8e3d726d-6f25-4356-9ebf-380d113f02ed-kube-api-access-dl4d8\") pod \"payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f\" (UID: \"8e3d726d-6f25-4356-9ebf-380d113f02ed\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:17:46.496513 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.496390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgr47\" (UniqueName: \"kubernetes.io/projected/1b1796bb-f4fd-491e-b48e-0bdbd6447870-kube-api-access-hgr47\") pod \"payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds\" (UID: \"1b1796bb-f4fd-491e-b48e-0bdbd6447870\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:17:46.496513 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.496474    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e3d726d-6f25-4356-9ebf-380d113f02ed-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f\" (UID: \"8e3d726d-6f25-4356-9ebf-380d113f02ed\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:17:46.496602 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.496532    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b1796bb-f4fd-491e-b48e-0bdbd6447870-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds\" (UID: \"1b1796bb-f4fd-491e-b48e-0bdbd6447870\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:17:46.497164 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.497148    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b1796bb-f4fd-491e-b48e-0bdbd6447870-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds\" (UID: \"1b1796bb-f4fd-491e-b48e-0bdbd6447870\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:17:46.504126 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.504106    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgr47\" (UniqueName: \"kubernetes.io/projected/1b1796bb-f4fd-491e-b48e-0bdbd6447870-kube-api-access-hgr47\") pod \"payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds\" (UID: \"1b1796bb-f4fd-491e-b48e-0bdbd6447870\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:17:46.597702 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.597677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e3d726d-6f25-4356-9ebf-380d113f02ed-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f\" (UID: \"8e3d726d-6f25-4356-9ebf-380d113f02ed\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:17:46.597792 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.597712    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl4d8\" (UniqueName: \"kubernetes.io/projected/8e3d726d-6f25-4356-9ebf-380d113f02ed-kube-api-access-dl4d8\") pod \"payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f\" (UID: \"8e3d726d-6f25-4356-9ebf-380d113f02ed\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:17:46.598319 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.598300    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e3d726d-6f25-4356-9ebf-380d113f02ed-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f\" (UID: \"8e3d726d-6f25-4356-9ebf-380d113f02ed\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:17:46.604419 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.604399    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl4d8\" (UniqueName: \"kubernetes.io/projected/8e3d726d-6f25-4356-9ebf-380d113f02ed-kube-api-access-dl4d8\") pod \"payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f\" (UID: \"8e3d726d-6f25-4356-9ebf-380d113f02ed\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:17:46.674398 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.674377    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:17:46.716280 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.716254    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:17:46.809280 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.809251    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"]
Jul 25 01:17:46.811613 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:17:46.811578    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b1796bb_f4fd_491e_b48e_0bdbd6447870.slice/crio-4470ce6c28678b293bf3b4d20fb34bc9ca13a69184d2a7fcd5e60790da62ffaa WatchSource:0}: Error finding container 4470ce6c28678b293bf3b4d20fb34bc9ca13a69184d2a7fcd5e60790da62ffaa: Status 404 returned error can't find the container with id 4470ce6c28678b293bf3b4d20fb34bc9ca13a69184d2a7fcd5e60790da62ffaa
Jul 25 01:17:46.872733 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:46.872710    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"]
Jul 25 01:17:46.875230 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:17:46.875202    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e3d726d_6f25_4356_9ebf_380d113f02ed.slice/crio-ba41c67103873384e274575f7900fc5cbec8ba35fd66d5e2070b848c922e3a02 WatchSource:0}: Error finding container ba41c67103873384e274575f7900fc5cbec8ba35fd66d5e2070b848c922e3a02: Status 404 returned error can't find the container with id ba41c67103873384e274575f7900fc5cbec8ba35fd66d5e2070b848c922e3a02
Jul 25 01:17:47.080413 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:47.080325    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds" event={"ID":"1b1796bb-f4fd-491e-b48e-0bdbd6447870","Type":"ContainerStarted","Data":"2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32"}
Jul 25 01:17:47.080413 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:47.080386    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:17:47.080413 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:47.080410    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds" event={"ID":"1b1796bb-f4fd-491e-b48e-0bdbd6447870","Type":"ContainerStarted","Data":"4470ce6c28678b293bf3b4d20fb34bc9ca13a69184d2a7fcd5e60790da62ffaa"}
Jul 25 01:17:47.081787 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:47.081765    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f" event={"ID":"8e3d726d-6f25-4356-9ebf-380d113f02ed","Type":"ContainerStarted","Data":"5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8"}
Jul 25 01:17:47.081908 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:47.081801    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f" event={"ID":"8e3d726d-6f25-4356-9ebf-380d113f02ed","Type":"ContainerStarted","Data":"ba41c67103873384e274575f7900fc5cbec8ba35fd66d5e2070b848c922e3a02"}
Jul 25 01:17:47.081908 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:47.081886    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:17:47.097091 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:47.097032    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds" podStartSLOduration=1.097020165 podStartE2EDuration="1.097020165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:17:47.096703253 +0000 UTC m=+2489.485225656" watchObservedRunningTime="2026-07-25 01:17:47.097020165 +0000 UTC m=+2489.485542546"
Jul 25 01:17:47.115942 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:47.115900    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f" podStartSLOduration=1.115887473 podStartE2EDuration="1.115887473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:17:47.114674531 +0000 UTC m=+2489.503196946" watchObservedRunningTime="2026-07-25 01:17:47.115887473 +0000 UTC m=+2489.504409875"
Jul 25 01:17:52.690608 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:52.690568    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:17:54.898553 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.898520    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"]
Jul 25 01:17:54.901767 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.901751    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:17:54.903834 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.903812    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-9deb1f\""
Jul 25 01:17:54.903974 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.903925    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-9deb1f-dockercfg-jtbzm\""
Jul 25 01:17:54.906432 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.906408    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"]
Jul 25 01:17:54.909347 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.909329    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:17:54.914098 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.914049    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"]
Jul 25 01:17:54.932483 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.932461    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"]
Jul 25 01:17:54.954601 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.954576    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd9pr\" (UniqueName: \"kubernetes.io/projected/b8b71f45-d41f-4120-a0e7-8122f1c24479-kube-api-access-qd9pr\") pod \"payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j\" (UID: \"b8b71f45-d41f-4120-a0e7-8122f1c24479\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:17:54.954697 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.954605    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b71f45-d41f-4120-a0e7-8122f1c24479-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j\" (UID: \"b8b71f45-d41f-4120-a0e7-8122f1c24479\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:17:54.954697 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.954642    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09701118-8df3-412b-bb76-ab97ce2d4d13-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb\" (UID: \"09701118-8df3-412b-bb76-ab97ce2d4d13\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:17:54.954807 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:54.954705    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqg6z\" (UniqueName: \"kubernetes.io/projected/09701118-8df3-412b-bb76-ab97ce2d4d13-kube-api-access-jqg6z\") pod \"payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb\" (UID: \"09701118-8df3-412b-bb76-ab97ce2d4d13\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:17:55.055129 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.055094    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd9pr\" (UniqueName: \"kubernetes.io/projected/b8b71f45-d41f-4120-a0e7-8122f1c24479-kube-api-access-qd9pr\") pod \"payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j\" (UID: \"b8b71f45-d41f-4120-a0e7-8122f1c24479\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:17:55.055129 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.055131    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b71f45-d41f-4120-a0e7-8122f1c24479-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j\" (UID: \"b8b71f45-d41f-4120-a0e7-8122f1c24479\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:17:55.055348 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.055196    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09701118-8df3-412b-bb76-ab97ce2d4d13-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb\" (UID: \"09701118-8df3-412b-bb76-ab97ce2d4d13\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:17:55.055348 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.055231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqg6z\" (UniqueName: \"kubernetes.io/projected/09701118-8df3-412b-bb76-ab97ce2d4d13-kube-api-access-jqg6z\") pod \"payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb\" (UID: \"09701118-8df3-412b-bb76-ab97ce2d4d13\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:17:55.055859 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.055842    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b71f45-d41f-4120-a0e7-8122f1c24479-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j\" (UID: \"b8b71f45-d41f-4120-a0e7-8122f1c24479\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:17:55.055897 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.055835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09701118-8df3-412b-bb76-ab97ce2d4d13-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb\" (UID: \"09701118-8df3-412b-bb76-ab97ce2d4d13\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:17:55.062800 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.062774    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqg6z\" (UniqueName: \"kubernetes.io/projected/09701118-8df3-412b-bb76-ab97ce2d4d13-kube-api-access-jqg6z\") pod \"payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb\" (UID: \"09701118-8df3-412b-bb76-ab97ce2d4d13\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:17:55.062895 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.062839    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd9pr\" (UniqueName: \"kubernetes.io/projected/b8b71f45-d41f-4120-a0e7-8122f1c24479-kube-api-access-qd9pr\") pod \"payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j\" (UID: \"b8b71f45-d41f-4120-a0e7-8122f1c24479\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:17:55.212437 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.212355    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:17:55.221481 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.221456    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:17:55.346166 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.346133    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"]
Jul 25 01:17:55.349560 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:17:55.349527    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09701118_8df3_412b_bb76_ab97ce2d4d13.slice/crio-3aa61c8d0b9bb92df26a7a893ee03bfd4f7d0359b5c5f696dcf3e89ac2489e1d WatchSource:0}: Error finding container 3aa61c8d0b9bb92df26a7a893ee03bfd4f7d0359b5c5f696dcf3e89ac2489e1d: Status 404 returned error can't find the container with id 3aa61c8d0b9bb92df26a7a893ee03bfd4f7d0359b5c5f696dcf3e89ac2489e1d
Jul 25 01:17:55.369476 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:55.369453    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"]
Jul 25 01:17:55.382098 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:17:55.382053    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b71f45_d41f_4120_a0e7_8122f1c24479.slice/crio-dfa4c0155437db2d2352506bd380be0c9034e59adcb05afeb19d9941f6f76c27 WatchSource:0}: Error finding container dfa4c0155437db2d2352506bd380be0c9034e59adcb05afeb19d9941f6f76c27: Status 404 returned error can't find the container with id dfa4c0155437db2d2352506bd380be0c9034e59adcb05afeb19d9941f6f76c27
Jul 25 01:17:56.121787 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:56.121753    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb" event={"ID":"09701118-8df3-412b-bb76-ab97ce2d4d13","Type":"ContainerStarted","Data":"caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625"}
Jul 25 01:17:56.122194 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:56.121821    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:17:56.122194 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:56.121849    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb" event={"ID":"09701118-8df3-412b-bb76-ab97ce2d4d13","Type":"ContainerStarted","Data":"3aa61c8d0b9bb92df26a7a893ee03bfd4f7d0359b5c5f696dcf3e89ac2489e1d"}
Jul 25 01:17:56.123297 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:56.123269    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j" event={"ID":"b8b71f45-d41f-4120-a0e7-8122f1c24479","Type":"ContainerStarted","Data":"ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe"}
Jul 25 01:17:56.123427 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:56.123319    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j" event={"ID":"b8b71f45-d41f-4120-a0e7-8122f1c24479","Type":"ContainerStarted","Data":"dfa4c0155437db2d2352506bd380be0c9034e59adcb05afeb19d9941f6f76c27"}
Jul 25 01:17:56.123427 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:56.123347    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:17:56.157084 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:56.157027    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb" podStartSLOduration=2.157014 podStartE2EDuration="2.157014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:17:56.155130068 +0000 UTC m=+2498.543652471" watchObservedRunningTime="2026-07-25 01:17:56.157014 +0000 UTC m=+2498.545536406"
Jul 25 01:17:56.178642 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:56.178606    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j" podStartSLOduration=2.178591177 podStartE2EDuration="2.178591177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:17:56.177503953 +0000 UTC m=+2498.566026355" watchObservedRunningTime="2026-07-25 01:17:56.178591177 +0000 UTC m=+2498.567113579"
Jul 25 01:17:58.088462 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:58.088434    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:17:58.088831 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:17:58.088507    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:18:06.094035 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:06.093998    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:18:07.130851 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:07.130821    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:18:07.131286 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:07.130897    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:18:13.783330 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.783255    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"]
Jul 25 01:18:13.786969 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.786952    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:13.789316 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.789295    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d08be1\"/\"openshift-service-ca.crt\""
Jul 25 01:18:13.789427 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.789295    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d08be1\"/\"kube-root-ca.crt\""
Jul 25 01:18:13.790032 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.790010    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d08be1\"/\"default-dockercfg-hl8ms\""
Jul 25 01:18:13.790172 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.790121    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d08be1\"/\"test-model-d08be1-kserve-self-signed-certs\""
Jul 25 01:18:13.800473 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.800452    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"]
Jul 25 01:18:13.914692 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.914666    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0679a31d-d455-4da8-b751-aaca244bc1f5-tls-certs\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:13.914841 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.914709    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr6zs\" (UniqueName: \"kubernetes.io/projected/0679a31d-d455-4da8-b751-aaca244bc1f5-kube-api-access-gr6zs\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:13.914841 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.914739    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-model-cache\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:13.914841 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.914812    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-dshm\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:13.914965 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.914858    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-tmp-dir\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:13.914965 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:13.914888    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-home\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.016171 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.016138    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0679a31d-d455-4da8-b751-aaca244bc1f5-tls-certs\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.016324 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.016181    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr6zs\" (UniqueName: \"kubernetes.io/projected/0679a31d-d455-4da8-b751-aaca244bc1f5-kube-api-access-gr6zs\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.016324 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.016208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-model-cache\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.016324 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.016235    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-dshm\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.016476 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.016339    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-tmp-dir\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.016476 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.016379    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-home\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.016689 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.016671    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-model-cache\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.016757 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.016734    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-home\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.016807 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.016753    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-tmp-dir\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.018390 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.018368    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-dshm\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.018510 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.018495    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0679a31d-d455-4da8-b751-aaca244bc1f5-tls-certs\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.024263 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.024240    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr6zs\" (UniqueName: \"kubernetes.io/projected/0679a31d-d455-4da8-b751-aaca244bc1f5-kube-api-access-gr6zs\") pod \"test-model-d08be1-kserve-7c69675785-pvfqq\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") " pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.096009 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.095940    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:14.218690 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:14.218654    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"]
Jul 25 01:18:14.221596 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:18:14.221556    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0679a31d_d455_4da8_b751_aaca244bc1f5.slice/crio-ec3af3c9df8e33671cfd3086cb9da56823e25bdc3479b7ba71d517eed77c1b40 WatchSource:0}: Error finding container ec3af3c9df8e33671cfd3086cb9da56823e25bdc3479b7ba71d517eed77c1b40: Status 404 returned error can't find the container with id ec3af3c9df8e33671cfd3086cb9da56823e25bdc3479b7ba71d517eed77c1b40
Jul 25 01:18:15.204474 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:15.204431    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq" event={"ID":"0679a31d-d455-4da8-b751-aaca244bc1f5","Type":"ContainerStarted","Data":"5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb"}
Jul 25 01:18:15.204474 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:15.204473    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq" event={"ID":"0679a31d-d455-4da8-b751-aaca244bc1f5","Type":"ContainerStarted","Data":"ec3af3c9df8e33671cfd3086cb9da56823e25bdc3479b7ba71d517eed77c1b40"}
Jul 25 01:18:15.221594 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:15.221543    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq" podStartSLOduration=2.221527717 podStartE2EDuration="2.221527717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:15.220109421 +0000 UTC m=+2517.608631825" watchObservedRunningTime="2026-07-25 01:18:15.221527717 +0000 UTC m=+2517.610050118"
Jul 25 01:18:18.258670 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:18.258639    2566 scope.go:117] "RemoveContainer" containerID="f3477c21c58ff2c38966bc07a8f7517c7a74c39448438135b27a9096bb7aab2d"
Jul 25 01:18:19.405352 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:19.405319    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:18:24.096745 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:24.096705    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:24.097241 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:24.096758    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:24.108639 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:24.108613    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:24.252905 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:24.252883    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:18:41.688700 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:41.688656    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:18:56.085101 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:18:56.085045    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:19:18.501187 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:18.501148    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"]
Jul 25 01:19:23.085443 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:23.085407    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:19:30.139581 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.139544    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"]
Jul 25 01:19:30.140019 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.139795    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f" podUID="8e3d726d-6f25-4356-9ebf-380d113f02ed" containerName="payload-processing" containerID="cri-o://5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8" gracePeriod=30
Jul 25 01:19:30.147407 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.147380    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"]
Jul 25 01:19:30.147602 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.147579    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds" podUID="1b1796bb-f4fd-491e-b48e-0bdbd6447870" containerName="payload-pre-processing" containerID="cri-o://2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32" gracePeriod=30
Jul 25 01:19:30.419910 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.419886    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:19:30.422750 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.422730    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:19:30.517410 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.517376    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b1796bb-f4fd-491e-b48e-0bdbd6447870-plugins-config-volume\") pod \"1b1796bb-f4fd-491e-b48e-0bdbd6447870\" (UID: \"1b1796bb-f4fd-491e-b48e-0bdbd6447870\") "
Jul 25 01:19:30.517626 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.517454    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgr47\" (UniqueName: \"kubernetes.io/projected/1b1796bb-f4fd-491e-b48e-0bdbd6447870-kube-api-access-hgr47\") pod \"1b1796bb-f4fd-491e-b48e-0bdbd6447870\" (UID: \"1b1796bb-f4fd-491e-b48e-0bdbd6447870\") "
Jul 25 01:19:30.517626 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.517479    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl4d8\" (UniqueName: \"kubernetes.io/projected/8e3d726d-6f25-4356-9ebf-380d113f02ed-kube-api-access-dl4d8\") pod \"8e3d726d-6f25-4356-9ebf-380d113f02ed\" (UID: \"8e3d726d-6f25-4356-9ebf-380d113f02ed\") "
Jul 25 01:19:30.517626 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.517527    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e3d726d-6f25-4356-9ebf-380d113f02ed-plugins-config-volume\") pod \"8e3d726d-6f25-4356-9ebf-380d113f02ed\" (UID: \"8e3d726d-6f25-4356-9ebf-380d113f02ed\") "
Jul 25 01:19:30.517875 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.517850    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b1796bb-f4fd-491e-b48e-0bdbd6447870-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b1796bb-f4fd-491e-b48e-0bdbd6447870" (UID: "1b1796bb-f4fd-491e-b48e-0bdbd6447870"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:30.517936 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.517858    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3d726d-6f25-4356-9ebf-380d113f02ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e3d726d-6f25-4356-9ebf-380d113f02ed" (UID: "8e3d726d-6f25-4356-9ebf-380d113f02ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:30.519622 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.519591    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b1796bb-f4fd-491e-b48e-0bdbd6447870-kube-api-access-hgr47" (OuterVolumeSpecName: "kube-api-access-hgr47") pod "1b1796bb-f4fd-491e-b48e-0bdbd6447870" (UID: "1b1796bb-f4fd-491e-b48e-0bdbd6447870"). InnerVolumeSpecName "kube-api-access-hgr47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:19:30.519769 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.519627    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e3d726d-6f25-4356-9ebf-380d113f02ed-kube-api-access-dl4d8" (OuterVolumeSpecName: "kube-api-access-dl4d8") pod "8e3d726d-6f25-4356-9ebf-380d113f02ed" (UID: "8e3d726d-6f25-4356-9ebf-380d113f02ed"). InnerVolumeSpecName "kube-api-access-dl4d8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:19:30.543215 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.543188    2566 generic.go:356] "Generic (PLEG): container finished" podID="8e3d726d-6f25-4356-9ebf-380d113f02ed" containerID="5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8" exitCode=0
Jul 25 01:19:30.543311 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.543229    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"
Jul 25 01:19:30.543311 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.543277    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f" event={"ID":"8e3d726d-6f25-4356-9ebf-380d113f02ed","Type":"ContainerDied","Data":"5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8"}
Jul 25 01:19:30.543408 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.543356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f" event={"ID":"8e3d726d-6f25-4356-9ebf-380d113f02ed","Type":"ContainerDied","Data":"ba41c67103873384e274575f7900fc5cbec8ba35fd66d5e2070b848c922e3a02"}
Jul 25 01:19:30.543408 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.543387    2566 scope.go:117] "RemoveContainer" containerID="5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8"
Jul 25 01:19:30.544486 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.544468    2566 generic.go:356] "Generic (PLEG): container finished" podID="1b1796bb-f4fd-491e-b48e-0bdbd6447870" containerID="2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32" exitCode=0
Jul 25 01:19:30.544566 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.544525    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"
Jul 25 01:19:30.544566 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.544532    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds" event={"ID":"1b1796bb-f4fd-491e-b48e-0bdbd6447870","Type":"ContainerDied","Data":"2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32"}
Jul 25 01:19:30.544652 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.544567    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds" event={"ID":"1b1796bb-f4fd-491e-b48e-0bdbd6447870","Type":"ContainerDied","Data":"4470ce6c28678b293bf3b4d20fb34bc9ca13a69184d2a7fcd5e60790da62ffaa"}
Jul 25 01:19:30.552179 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.552161    2566 scope.go:117] "RemoveContainer" containerID="5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8"
Jul 25 01:19:30.552436 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:19:30.552416    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8\": container with ID starting with 5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8 not found: ID does not exist" containerID="5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8"
Jul 25 01:19:30.552496 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.552448    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8"} err="failed to get container status \"5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8\": rpc error: code = NotFound desc = could not find container \"5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8\": container with ID starting with 5ccff5cfd460e4073f3b5a80d4939412a6ac92249da60669287d2603fb3faef8 not found: ID does not exist"
Jul 25 01:19:30.552496 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.552471    2566 scope.go:117] "RemoveContainer" containerID="2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32"
Jul 25 01:19:30.559527 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.559511    2566 scope.go:117] "RemoveContainer" containerID="2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32"
Jul 25 01:19:30.559769 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:19:30.559755    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32\": container with ID starting with 2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32 not found: ID does not exist" containerID="2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32"
Jul 25 01:19:30.559808 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.559776    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32"} err="failed to get container status \"2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32\": rpc error: code = NotFound desc = could not find container \"2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32\": container with ID starting with 2269ab14931b8cc20c8b653931a72a40ae748e057074f6087f2fc1fa6d32ec32 not found: ID does not exist"
Jul 25 01:19:30.563682 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.563662    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"]
Jul 25 01:19:30.567419 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.567401    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d08be1-668f9bc6c8-9279f"]
Jul 25 01:19:30.575977 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.575953    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"]
Jul 25 01:19:30.579738 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.579718    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d08be1-56667566cb-zfpds"]
Jul 25 01:19:30.618367 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.618335    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgr47\" (UniqueName: \"kubernetes.io/projected/1b1796bb-f4fd-491e-b48e-0bdbd6447870-kube-api-access-hgr47\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:30.618367 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.618366    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl4d8\" (UniqueName: \"kubernetes.io/projected/8e3d726d-6f25-4356-9ebf-380d113f02ed-kube-api-access-dl4d8\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:30.618367 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.618376    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e3d726d-6f25-4356-9ebf-380d113f02ed-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:30.618513 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:30.618388    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b1796bb-f4fd-491e-b48e-0bdbd6447870-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:32.171025 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:32.170993    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b1796bb-f4fd-491e-b48e-0bdbd6447870" path="/var/lib/kubelet/pods/1b1796bb-f4fd-491e-b48e-0bdbd6447870/volumes"
Jul 25 01:19:32.171381 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:32.171334    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e3d726d-6f25-4356-9ebf-380d113f02ed" path="/var/lib/kubelet/pods/8e3d726d-6f25-4356-9ebf-380d113f02ed/volumes"
Jul 25 01:19:33.520828 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.520789    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq" podUID="0679a31d-d455-4da8-b751-aaca244bc1f5" containerName="main" containerID="cri-o://5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb" gracePeriod=45
Jul 25 01:19:33.770917 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.770867    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:19:33.943801 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.943772    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr6zs\" (UniqueName: \"kubernetes.io/projected/0679a31d-d455-4da8-b751-aaca244bc1f5-kube-api-access-gr6zs\") pod \"0679a31d-d455-4da8-b751-aaca244bc1f5\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") "
Jul 25 01:19:33.943933 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.943818    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-dshm\") pod \"0679a31d-d455-4da8-b751-aaca244bc1f5\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") "
Jul 25 01:19:33.943933 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.943852    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0679a31d-d455-4da8-b751-aaca244bc1f5-tls-certs\") pod \"0679a31d-d455-4da8-b751-aaca244bc1f5\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") "
Jul 25 01:19:33.943933 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.943908    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-model-cache\") pod \"0679a31d-d455-4da8-b751-aaca244bc1f5\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") "
Jul 25 01:19:33.943933 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.943932    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-home\") pod \"0679a31d-d455-4da8-b751-aaca244bc1f5\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") "
Jul 25 01:19:33.944163 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.943964    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-tmp-dir\") pod \"0679a31d-d455-4da8-b751-aaca244bc1f5\" (UID: \"0679a31d-d455-4da8-b751-aaca244bc1f5\") "
Jul 25 01:19:33.944291 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.944256    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "0679a31d-d455-4da8-b751-aaca244bc1f5" (UID: "0679a31d-d455-4da8-b751-aaca244bc1f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:33.944414 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.944264    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-home" (OuterVolumeSpecName: "home") pod "0679a31d-d455-4da8-b751-aaca244bc1f5" (UID: "0679a31d-d455-4da8-b751-aaca244bc1f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:33.944414 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.944304    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0679a31d-d455-4da8-b751-aaca244bc1f5" (UID: "0679a31d-d455-4da8-b751-aaca244bc1f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:33.946350 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.946326    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0679a31d-d455-4da8-b751-aaca244bc1f5-kube-api-access-gr6zs" (OuterVolumeSpecName: "kube-api-access-gr6zs") pod "0679a31d-d455-4da8-b751-aaca244bc1f5" (UID: "0679a31d-d455-4da8-b751-aaca244bc1f5"). InnerVolumeSpecName "kube-api-access-gr6zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:19:33.946464 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.946384    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0679a31d-d455-4da8-b751-aaca244bc1f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0679a31d-d455-4da8-b751-aaca244bc1f5" (UID: "0679a31d-d455-4da8-b751-aaca244bc1f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:19:33.946464 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:33.946384    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-dshm" (OuterVolumeSpecName: "dshm") pod "0679a31d-d455-4da8-b751-aaca244bc1f5" (UID: "0679a31d-d455-4da8-b751-aaca244bc1f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:34.044980 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.044918    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.044980 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.044949    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.044980 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.044958    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.044980 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.044967    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr6zs\" (UniqueName: \"kubernetes.io/projected/0679a31d-d455-4da8-b751-aaca244bc1f5-kube-api-access-gr6zs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.044980 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.044976    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0679a31d-d455-4da8-b751-aaca244bc1f5-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.044980 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.044984    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0679a31d-d455-4da8-b751-aaca244bc1f5-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.566410 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.566377    2566 generic.go:356] "Generic (PLEG): container finished" podID="0679a31d-d455-4da8-b751-aaca244bc1f5" containerID="5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb" exitCode=0
Jul 25 01:19:34.566876 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.566426    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"
Jul 25 01:19:34.566876 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.566447    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq" event={"ID":"0679a31d-d455-4da8-b751-aaca244bc1f5","Type":"ContainerDied","Data":"5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb"}
Jul 25 01:19:34.566876 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.566481    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq" event={"ID":"0679a31d-d455-4da8-b751-aaca244bc1f5","Type":"ContainerDied","Data":"ec3af3c9df8e33671cfd3086cb9da56823e25bdc3479b7ba71d517eed77c1b40"}
Jul 25 01:19:34.566876 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.566503    2566 scope.go:117] "RemoveContainer" containerID="5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb"
Jul 25 01:19:34.575337 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.575318    2566 scope.go:117] "RemoveContainer" containerID="5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb"
Jul 25 01:19:34.575597 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:19:34.575570    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb\": container with ID starting with 5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb not found: ID does not exist" containerID="5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb"
Jul 25 01:19:34.575663 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.575609    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb"} err="failed to get container status \"5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb\": rpc error: code = NotFound desc = could not find container \"5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb\": container with ID starting with 5144970ef55cd58f46a9a6e2db91dd13bb0f86b27a6c71491f69e7773f9016bb not found: ID does not exist"
Jul 25 01:19:34.580202 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.580176    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"]
Jul 25 01:19:34.584600 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:34.584578    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d08be1/test-model-d08be1-kserve-7c69675785-pvfqq"]
Jul 25 01:19:35.589511 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:35.589467    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:19:36.171887 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:36.171852    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0679a31d-d455-4da8-b751-aaca244bc1f5" path="/var/lib/kubelet/pods/0679a31d-d455-4da8-b751-aaca244bc1f5/volumes"
Jul 25 01:19:41.590975 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.590879    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"]
Jul 25 01:19:41.591355 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.591223    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b1796bb-f4fd-491e-b48e-0bdbd6447870" containerName="payload-pre-processing"
Jul 25 01:19:41.591355 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.591240    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b1796bb-f4fd-491e-b48e-0bdbd6447870" containerName="payload-pre-processing"
Jul 25 01:19:41.591355 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.591252    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3d726d-6f25-4356-9ebf-380d113f02ed" containerName="payload-processing"
Jul 25 01:19:41.591355 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.591261    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3d726d-6f25-4356-9ebf-380d113f02ed" containerName="payload-processing"
Jul 25 01:19:41.591355 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.591286    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0679a31d-d455-4da8-b751-aaca244bc1f5" containerName="main"
Jul 25 01:19:41.591355 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.591295    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0679a31d-d455-4da8-b751-aaca244bc1f5" containerName="main"
Jul 25 01:19:41.591534 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.591373    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b1796bb-f4fd-491e-b48e-0bdbd6447870" containerName="payload-pre-processing"
Jul 25 01:19:41.591534 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.591383    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0679a31d-d455-4da8-b751-aaca244bc1f5" containerName="main"
Jul 25 01:19:41.591534 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.591390    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3d726d-6f25-4356-9ebf-380d113f02ed" containerName="payload-processing"
Jul 25 01:19:41.595773 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.595753    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"
Jul 25 01:19:41.597922 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.597890    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:19:41.598094 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.597933    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:19:41.598169 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.598089    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d183572-0a2e-45c4-937e-6a44f171cc27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj\" (UID: \"4d183572-0a2e-45c4-937e-6a44f171cc27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"
Jul 25 01:19:41.600887 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.600860    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"]
Jul 25 01:19:41.699419 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.699390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d183572-0a2e-45c4-937e-6a44f171cc27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj\" (UID: \"4d183572-0a2e-45c4-937e-6a44f171cc27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"
Jul 25 01:19:41.700007 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.699979    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d183572-0a2e-45c4-937e-6a44f171cc27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj\" (UID: \"4d183572-0a2e-45c4-937e-6a44f171cc27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"
Jul 25 01:19:41.909577 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:41.909543    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"
Jul 25 01:19:42.037894 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:42.037867    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"]
Jul 25 01:19:42.039929 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:19:42.039900    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d183572_0a2e_45c4_937e_6a44f171cc27.slice/crio-eec128f63f6246baa82fcb062ca4356ab945c0efbb537742c581cddb4a5599db WatchSource:0}: Error finding container eec128f63f6246baa82fcb062ca4356ab945c0efbb537742c581cddb4a5599db: Status 404 returned error can't find the container with id eec128f63f6246baa82fcb062ca4356ab945c0efbb537742c581cddb4a5599db
Jul 25 01:19:42.606085 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:42.605997    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj" event={"ID":"4d183572-0a2e-45c4-937e-6a44f171cc27","Type":"ContainerStarted","Data":"f02b3a0e63993ba7176bb6bc514dfd1e83ce71eef2def8444b7c1adabd344751"}
Jul 25 01:19:42.606401 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:42.606087    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj" event={"ID":"4d183572-0a2e-45c4-937e-6a44f171cc27","Type":"ContainerStarted","Data":"eec128f63f6246baa82fcb062ca4356ab945c0efbb537742c581cddb4a5599db"}
Jul 25 01:19:42.625081 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:42.622395    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj" podStartSLOduration=1.62237539 podStartE2EDuration="1.62237539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:19:42.619102725 +0000 UTC m=+2605.007625127" watchObservedRunningTime="2026-07-25 01:19:42.62237539 +0000 UTC m=+2605.010897794"
Jul 25 01:19:43.612804 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:43.612769    2566 generic.go:356] "Generic (PLEG): container finished" podID="4d183572-0a2e-45c4-937e-6a44f171cc27" containerID="f02b3a0e63993ba7176bb6bc514dfd1e83ce71eef2def8444b7c1adabd344751" exitCode=0
Jul 25 01:19:43.613222 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:43.612857    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj" event={"ID":"4d183572-0a2e-45c4-937e-6a44f171cc27","Type":"ContainerDied","Data":"f02b3a0e63993ba7176bb6bc514dfd1e83ce71eef2def8444b7c1adabd344751"}
Jul 25 01:19:44.762678 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:44.762654    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"
Jul 25 01:19:44.818340 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:44.818314    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d183572-0a2e-45c4-937e-6a44f171cc27-maas-api-service-ca\") pod \"4d183572-0a2e-45c4-937e-6a44f171cc27\" (UID: \"4d183572-0a2e-45c4-937e-6a44f171cc27\") "
Jul 25 01:19:44.818679 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:44.818658    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d183572-0a2e-45c4-937e-6a44f171cc27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d183572-0a2e-45c4-937e-6a44f171cc27" (UID: "4d183572-0a2e-45c4-937e-6a44f171cc27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:44.919766 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:44.919713    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d183572-0a2e-45c4-937e-6a44f171cc27-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:45.623722 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:45.623685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj" event={"ID":"4d183572-0a2e-45c4-937e-6a44f171cc27","Type":"ContainerDied","Data":"eec128f63f6246baa82fcb062ca4356ab945c0efbb537742c581cddb4a5599db"}
Jul 25 01:19:45.623850 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:45.623737    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eec128f63f6246baa82fcb062ca4356ab945c0efbb537742c581cddb4a5599db"
Jul 25 01:19:45.623850 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:45.623733    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"
Jul 25 01:19:52.068993 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.068958    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"]
Jul 25 01:19:52.069450 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.069216    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j" podUID="b8b71f45-d41f-4120-a0e7-8122f1c24479" containerName="payload-processing" containerID="cri-o://ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe" gracePeriod=30
Jul 25 01:19:52.087305 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.087282    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"]
Jul 25 01:19:52.087545 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.087506    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb" podUID="09701118-8df3-412b-bb76-ab97ce2d4d13" containerName="payload-pre-processing" containerID="cri-o://caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625" gracePeriod=30
Jul 25 01:19:52.348630 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.348603    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:19:52.351209 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.351188    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:19:52.372214 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.372194    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd9pr\" (UniqueName: \"kubernetes.io/projected/b8b71f45-d41f-4120-a0e7-8122f1c24479-kube-api-access-qd9pr\") pod \"b8b71f45-d41f-4120-a0e7-8122f1c24479\" (UID: \"b8b71f45-d41f-4120-a0e7-8122f1c24479\") "
Jul 25 01:19:52.372324 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.372241    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b71f45-d41f-4120-a0e7-8122f1c24479-plugins-config-volume\") pod \"b8b71f45-d41f-4120-a0e7-8122f1c24479\" (UID: \"b8b71f45-d41f-4120-a0e7-8122f1c24479\") "
Jul 25 01:19:52.372423 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.372401    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqg6z\" (UniqueName: \"kubernetes.io/projected/09701118-8df3-412b-bb76-ab97ce2d4d13-kube-api-access-jqg6z\") pod \"09701118-8df3-412b-bb76-ab97ce2d4d13\" (UID: \"09701118-8df3-412b-bb76-ab97ce2d4d13\") "
Jul 25 01:19:52.372522 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.372506    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09701118-8df3-412b-bb76-ab97ce2d4d13-plugins-config-volume\") pod \"09701118-8df3-412b-bb76-ab97ce2d4d13\" (UID: \"09701118-8df3-412b-bb76-ab97ce2d4d13\") "
Jul 25 01:19:52.372598 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.372511    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8b71f45-d41f-4120-a0e7-8122f1c24479-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8b71f45-d41f-4120-a0e7-8122f1c24479" (UID: "b8b71f45-d41f-4120-a0e7-8122f1c24479"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:52.372735 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.372708    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8b71f45-d41f-4120-a0e7-8122f1c24479-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:52.373008 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.372986    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09701118-8df3-412b-bb76-ab97ce2d4d13-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09701118-8df3-412b-bb76-ab97ce2d4d13" (UID: "09701118-8df3-412b-bb76-ab97ce2d4d13"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:52.374254 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.374236    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8b71f45-d41f-4120-a0e7-8122f1c24479-kube-api-access-qd9pr" (OuterVolumeSpecName: "kube-api-access-qd9pr") pod "b8b71f45-d41f-4120-a0e7-8122f1c24479" (UID: "b8b71f45-d41f-4120-a0e7-8122f1c24479"). InnerVolumeSpecName "kube-api-access-qd9pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:19:52.374325 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.374269    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09701118-8df3-412b-bb76-ab97ce2d4d13-kube-api-access-jqg6z" (OuterVolumeSpecName: "kube-api-access-jqg6z") pod "09701118-8df3-412b-bb76-ab97ce2d4d13" (UID: "09701118-8df3-412b-bb76-ab97ce2d4d13"). InnerVolumeSpecName "kube-api-access-jqg6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:19:52.473978 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.473905    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09701118-8df3-412b-bb76-ab97ce2d4d13-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:52.473978 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.473938    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd9pr\" (UniqueName: \"kubernetes.io/projected/b8b71f45-d41f-4120-a0e7-8122f1c24479-kube-api-access-qd9pr\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:52.473978 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.473948    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqg6z\" (UniqueName: \"kubernetes.io/projected/09701118-8df3-412b-bb76-ab97ce2d4d13-kube-api-access-jqg6z\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:19:52.658304 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.658276    2566 generic.go:356] "Generic (PLEG): container finished" podID="09701118-8df3-412b-bb76-ab97ce2d4d13" containerID="caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625" exitCode=0
Jul 25 01:19:52.658425 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.658328    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"
Jul 25 01:19:52.658425 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.658380    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb" event={"ID":"09701118-8df3-412b-bb76-ab97ce2d4d13","Type":"ContainerDied","Data":"caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625"}
Jul 25 01:19:52.658506 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.658442    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb" event={"ID":"09701118-8df3-412b-bb76-ab97ce2d4d13","Type":"ContainerDied","Data":"3aa61c8d0b9bb92df26a7a893ee03bfd4f7d0359b5c5f696dcf3e89ac2489e1d"}
Jul 25 01:19:52.658506 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.658469    2566 scope.go:117] "RemoveContainer" containerID="caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625"
Jul 25 01:19:52.659748 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.659724    2566 generic.go:356] "Generic (PLEG): container finished" podID="b8b71f45-d41f-4120-a0e7-8122f1c24479" containerID="ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe" exitCode=0
Jul 25 01:19:52.659861 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.659758    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j" event={"ID":"b8b71f45-d41f-4120-a0e7-8122f1c24479","Type":"ContainerDied","Data":"ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe"}
Jul 25 01:19:52.659861 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.659826    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j" event={"ID":"b8b71f45-d41f-4120-a0e7-8122f1c24479","Type":"ContainerDied","Data":"dfa4c0155437db2d2352506bd380be0c9034e59adcb05afeb19d9941f6f76c27"}
Jul 25 01:19:52.659861 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.659772    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"
Jul 25 01:19:52.668683 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.668662    2566 scope.go:117] "RemoveContainer" containerID="caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625"
Jul 25 01:19:52.669119 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:19:52.668967    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625\": container with ID starting with caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625 not found: ID does not exist" containerID="caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625"
Jul 25 01:19:52.669119 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.668995    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625"} err="failed to get container status \"caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625\": rpc error: code = NotFound desc = could not find container \"caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625\": container with ID starting with caffad37418b466bd0017f463822ac22584efaeae1b76bc7d42b857d0c6ed625 not found: ID does not exist"
Jul 25 01:19:52.669119 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.669014    2566 scope.go:117] "RemoveContainer" containerID="ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe"
Jul 25 01:19:52.679557 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.679534    2566 scope.go:117] "RemoveContainer" containerID="ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe"
Jul 25 01:19:52.679854 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:19:52.679829    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe\": container with ID starting with ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe not found: ID does not exist" containerID="ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe"
Jul 25 01:19:52.679955 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.679858    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe"} err="failed to get container status \"ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe\": rpc error: code = NotFound desc = could not find container \"ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe\": container with ID starting with ca53436eb4122c5c267ea3eb59af03f0b4313260dab6d6558d52ebb56f237dbe not found: ID does not exist"
Jul 25 01:19:52.685877 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.685854    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"]
Jul 25 01:19:52.689871 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.689847    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9deb1f-6f7fb7fcf6-fp9gb"]
Jul 25 01:19:52.700148 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.700123    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"]
Jul 25 01:19:52.707976 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:52.707952    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9deb1f-744d897dfd-mvp4j"]
Jul 25 01:19:54.170804 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:54.170751    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09701118-8df3-412b-bb76-ab97ce2d4d13" path="/var/lib/kubelet/pods/09701118-8df3-412b-bb76-ab97ce2d4d13/volumes"
Jul 25 01:19:54.171507 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:54.171479    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8b71f45-d41f-4120-a0e7-8122f1c24479" path="/var/lib/kubelet/pods/b8b71f45-d41f-4120-a0e7-8122f1c24479/volumes"
Jul 25 01:19:55.094905 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:55.094865    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:19:56.662244 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:56.662197    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"]
Jul 25 01:19:56.663469 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:56.663443    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9deb1f-mmgmj"]
Jul 25 01:19:58.171557 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:19:58.171519    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d183572-0a2e-45c4-937e-6a44f171cc27" path="/var/lib/kubelet/pods/4d183572-0a2e-45c4-937e-6a44f171cc27/volumes"
Jul 25 01:20:07.404964 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.404930    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"]
Jul 25 01:20:07.405350 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.405261    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d183572-0a2e-45c4-937e-6a44f171cc27" containerName="revoke-keys"
Jul 25 01:20:07.405350 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.405273    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d183572-0a2e-45c4-937e-6a44f171cc27" containerName="revoke-keys"
Jul 25 01:20:07.405350 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.405292    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b71f45-d41f-4120-a0e7-8122f1c24479" containerName="payload-processing"
Jul 25 01:20:07.405350 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.405298    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b71f45-d41f-4120-a0e7-8122f1c24479" containerName="payload-processing"
Jul 25 01:20:07.405350 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.405306    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09701118-8df3-412b-bb76-ab97ce2d4d13" containerName="payload-pre-processing"
Jul 25 01:20:07.405350 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.405312    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="09701118-8df3-412b-bb76-ab97ce2d4d13" containerName="payload-pre-processing"
Jul 25 01:20:07.405531 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.405362    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d183572-0a2e-45c4-937e-6a44f171cc27" containerName="revoke-keys"
Jul 25 01:20:07.405531 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.405369    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b71f45-d41f-4120-a0e7-8122f1c24479" containerName="payload-processing"
Jul 25 01:20:07.405531 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.405378    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="09701118-8df3-412b-bb76-ab97ce2d4d13" containerName="payload-pre-processing"
Jul 25 01:20:07.408435 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.408418    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:07.410666 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.410637    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-c59e95\""
Jul 25 01:20:07.410810 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.410735    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-c59e95-dockercfg-pgq5n\""
Jul 25 01:20:07.418818 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.418798    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"]
Jul 25 01:20:07.480486 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.480447    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed633317-af31-4e6d-9019-e4f62180c193-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq\" (UID: \"ed633317-af31-4e6d-9019-e4f62180c193\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:07.480656 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.480563    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp2dx\" (UniqueName: \"kubernetes.io/projected/ed633317-af31-4e6d-9019-e4f62180c193-kube-api-access-vp2dx\") pod \"payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq\" (UID: \"ed633317-af31-4e6d-9019-e4f62180c193\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:07.487320 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.487297    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"]
Jul 25 01:20:07.490234 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.490220    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:07.503230 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.503210    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"]
Jul 25 01:20:07.581278 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.581253    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp2dx\" (UniqueName: \"kubernetes.io/projected/ed633317-af31-4e6d-9019-e4f62180c193-kube-api-access-vp2dx\") pod \"payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq\" (UID: \"ed633317-af31-4e6d-9019-e4f62180c193\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:07.581377 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.581299    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljjrv\" (UniqueName: \"kubernetes.io/projected/2e82012a-df95-4d95-b57e-565e062c04db-kube-api-access-ljjrv\") pod \"payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb\" (UID: \"2e82012a-df95-4d95-b57e-565e062c04db\") " pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:07.581377 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.581351    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed633317-af31-4e6d-9019-e4f62180c193-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq\" (UID: \"ed633317-af31-4e6d-9019-e4f62180c193\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:07.581457 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.581377    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e82012a-df95-4d95-b57e-565e062c04db-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb\" (UID: \"2e82012a-df95-4d95-b57e-565e062c04db\") " pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:07.581997 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.581978    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed633317-af31-4e6d-9019-e4f62180c193-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq\" (UID: \"ed633317-af31-4e6d-9019-e4f62180c193\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:07.589925 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.589896    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp2dx\" (UniqueName: \"kubernetes.io/projected/ed633317-af31-4e6d-9019-e4f62180c193-kube-api-access-vp2dx\") pod \"payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq\" (UID: \"ed633317-af31-4e6d-9019-e4f62180c193\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:07.682524 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.682473    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e82012a-df95-4d95-b57e-565e062c04db-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb\" (UID: \"2e82012a-df95-4d95-b57e-565e062c04db\") " pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:07.682598 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.682527    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljjrv\" (UniqueName: \"kubernetes.io/projected/2e82012a-df95-4d95-b57e-565e062c04db-kube-api-access-ljjrv\") pod \"payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb\" (UID: \"2e82012a-df95-4d95-b57e-565e062c04db\") " pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:07.683151 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.683135    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e82012a-df95-4d95-b57e-565e062c04db-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb\" (UID: \"2e82012a-df95-4d95-b57e-565e062c04db\") " pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:07.690410 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.690388    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljjrv\" (UniqueName: \"kubernetes.io/projected/2e82012a-df95-4d95-b57e-565e062c04db-kube-api-access-ljjrv\") pod \"payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb\" (UID: \"2e82012a-df95-4d95-b57e-565e062c04db\") " pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:07.718645 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.718617    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:07.799482 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.799452    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:07.841038 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.841016    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"]
Jul 25 01:20:07.843109 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:20:07.843082    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded633317_af31_4e6d_9019_e4f62180c193.slice/crio-990530633ab5acaa39404d6a9b0e29c4a1fb6019d5d175004db6c025740c858c WatchSource:0}: Error finding container 990530633ab5acaa39404d6a9b0e29c4a1fb6019d5d175004db6c025740c858c: Status 404 returned error can't find the container with id 990530633ab5acaa39404d6a9b0e29c4a1fb6019d5d175004db6c025740c858c
Jul 25 01:20:07.923721 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:07.923701    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"]
Jul 25 01:20:07.926033 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:20:07.926009    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e82012a_df95_4d95_b57e_565e062c04db.slice/crio-b9cf6ca90671ba65d190c08fd9adbc7554d252f645afd7b40ad12ba74aa3b592 WatchSource:0}: Error finding container b9cf6ca90671ba65d190c08fd9adbc7554d252f645afd7b40ad12ba74aa3b592: Status 404 returned error can't find the container with id b9cf6ca90671ba65d190c08fd9adbc7554d252f645afd7b40ad12ba74aa3b592
Jul 25 01:20:08.743138 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:08.743099    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq" event={"ID":"ed633317-af31-4e6d-9019-e4f62180c193","Type":"ContainerStarted","Data":"0d54da1f9b16d02ccbe6fce7aa62b332dbb01954ba9f74041393812a6641d315"}
Jul 25 01:20:08.743537 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:08.743164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq" event={"ID":"ed633317-af31-4e6d-9019-e4f62180c193","Type":"ContainerStarted","Data":"990530633ab5acaa39404d6a9b0e29c4a1fb6019d5d175004db6c025740c858c"}
Jul 25 01:20:08.743537 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:08.743195    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:08.744592 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:08.744562    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb" event={"ID":"2e82012a-df95-4d95-b57e-565e062c04db","Type":"ContainerStarted","Data":"55d9bfc44dd8e8c60e002027fcfd8c95b2b69512de89e2c887adc8930d61196c"}
Jul 25 01:20:08.744709 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:08.744610    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb" event={"ID":"2e82012a-df95-4d95-b57e-565e062c04db","Type":"ContainerStarted","Data":"b9cf6ca90671ba65d190c08fd9adbc7554d252f645afd7b40ad12ba74aa3b592"}
Jul 25 01:20:08.744709 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:08.744636    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:08.760644 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:08.760598    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq" podStartSLOduration=1.760584908 podStartE2EDuration="1.760584908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:08.759173074 +0000 UTC m=+2631.147695477" watchObservedRunningTime="2026-07-25 01:20:08.760584908 +0000 UTC m=+2631.149107289"
Jul 25 01:20:08.775274 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:08.775229    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb" podStartSLOduration=1.775216118 podStartE2EDuration="1.775216118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:08.774755519 +0000 UTC m=+2631.163277922" watchObservedRunningTime="2026-07-25 01:20:08.775216118 +0000 UTC m=+2631.163738521"
Jul 25 01:20:14.190736 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:14.190699    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:20:15.887742 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:15.887712    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"]
Jul 25 01:20:15.891001 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:15.890983    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:20:15.893966 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:15.893942    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-015d51\""
Jul 25 01:20:15.894396 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:15.894380    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-015d51-dockercfg-5cwqm\""
Jul 25 01:20:15.911461 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:15.911440    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"]
Jul 25 01:20:15.938495 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:15.938474    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99a911a0-18a9-42c8-8046-89c849686124-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj\" (UID: \"99a911a0-18a9-42c8-8046-89c849686124\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:20:15.938591 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:15.938514    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmntb\" (UniqueName: \"kubernetes.io/projected/99a911a0-18a9-42c8-8046-89c849686124-kube-api-access-lmntb\") pod \"payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj\" (UID: \"99a911a0-18a9-42c8-8046-89c849686124\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:20:16.019153 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.019130    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"]
Jul 25 01:20:16.022305 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.022291    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:20:16.039214 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.039184    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99a911a0-18a9-42c8-8046-89c849686124-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj\" (UID: \"99a911a0-18a9-42c8-8046-89c849686124\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:20:16.039301 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.039241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmntb\" (UniqueName: \"kubernetes.io/projected/99a911a0-18a9-42c8-8046-89c849686124-kube-api-access-lmntb\") pod \"payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj\" (UID: \"99a911a0-18a9-42c8-8046-89c849686124\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:20:16.039794 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.039777    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99a911a0-18a9-42c8-8046-89c849686124-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj\" (UID: \"99a911a0-18a9-42c8-8046-89c849686124\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:20:16.043305 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.043287    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"]
Jul 25 01:20:16.057174 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.057155    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmntb\" (UniqueName: \"kubernetes.io/projected/99a911a0-18a9-42c8-8046-89c849686124-kube-api-access-lmntb\") pod \"payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj\" (UID: \"99a911a0-18a9-42c8-8046-89c849686124\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:20:16.139942 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.139894    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhxpz\" (UniqueName: \"kubernetes.io/projected/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-kube-api-access-nhxpz\") pod \"payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5\" (UID: \"a3cb5e87-963b-44b3-bdcc-f51d93365c2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:20:16.139942 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.139921    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5\" (UID: \"a3cb5e87-963b-44b3-bdcc-f51d93365c2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:20:16.201826 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.201806    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:20:16.241849 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.241247    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhxpz\" (UniqueName: \"kubernetes.io/projected/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-kube-api-access-nhxpz\") pod \"payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5\" (UID: \"a3cb5e87-963b-44b3-bdcc-f51d93365c2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:20:16.241849 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.241295    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5\" (UID: \"a3cb5e87-963b-44b3-bdcc-f51d93365c2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:20:16.242149 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.242130    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5\" (UID: \"a3cb5e87-963b-44b3-bdcc-f51d93365c2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:20:16.248893 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.248838    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhxpz\" (UniqueName: \"kubernetes.io/projected/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-kube-api-access-nhxpz\") pod \"payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5\" (UID: \"a3cb5e87-963b-44b3-bdcc-f51d93365c2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:20:16.330796 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.330770    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"]
Jul 25 01:20:16.330880 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.330808    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:20:16.333293 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:20:16.333271    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a911a0_18a9_42c8_8046_89c849686124.slice/crio-f82578453f617d61c876e49b975fa222d56b39fa48599319caf46775a75443fe WatchSource:0}: Error finding container f82578453f617d61c876e49b975fa222d56b39fa48599319caf46775a75443fe: Status 404 returned error can't find the container with id f82578453f617d61c876e49b975fa222d56b39fa48599319caf46775a75443fe
Jul 25 01:20:16.455426 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.455374    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"]
Jul 25 01:20:16.457393 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:20:16.457354    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3cb5e87_963b_44b3_bdcc_f51d93365c2d.slice/crio-89261406a924370f4a2c95ec6b91f993a132d55ccabf931d18a45d3d9c547822 WatchSource:0}: Error finding container 89261406a924370f4a2c95ec6b91f993a132d55ccabf931d18a45d3d9c547822: Status 404 returned error can't find the container with id 89261406a924370f4a2c95ec6b91f993a132d55ccabf931d18a45d3d9c547822
Jul 25 01:20:16.783257 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.783168    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5" event={"ID":"a3cb5e87-963b-44b3-bdcc-f51d93365c2d","Type":"ContainerStarted","Data":"26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c"}
Jul 25 01:20:16.783257 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.783237    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5" event={"ID":"a3cb5e87-963b-44b3-bdcc-f51d93365c2d","Type":"ContainerStarted","Data":"89261406a924370f4a2c95ec6b91f993a132d55ccabf931d18a45d3d9c547822"}
Jul 25 01:20:16.783454 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.783309    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:20:16.789646 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.789607    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj" event={"ID":"99a911a0-18a9-42c8-8046-89c849686124","Type":"ContainerStarted","Data":"a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8"}
Jul 25 01:20:16.789779 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.789686    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj" event={"ID":"99a911a0-18a9-42c8-8046-89c849686124","Type":"ContainerStarted","Data":"f82578453f617d61c876e49b975fa222d56b39fa48599319caf46775a75443fe"}
Jul 25 01:20:16.789779 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.789716    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:20:16.821437 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:16.821394    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5" podStartSLOduration=1.821382015 podStartE2EDuration="1.821382015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:16.818868521 +0000 UTC m=+2639.207390935" watchObservedRunningTime="2026-07-25 01:20:16.821382015 +0000 UTC m=+2639.209904488"
Jul 25 01:20:18.282577 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:18.282550    2566 scope.go:117] "RemoveContainer" containerID="111e86f0dc7ec0fa1a6dfae1df3eb279f15566e02c717aed5be652870b78a258"
Jul 25 01:20:19.751434 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:19.751401    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:19.751822 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:19.751470    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:19.769010 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:19.768956    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj" podStartSLOduration=4.7689408669999995 podStartE2EDuration="4.768940867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:16.846155822 +0000 UTC m=+2639.234678226" watchObservedRunningTime="2026-07-25 01:20:19.768940867 +0000 UTC m=+2642.157463269"
Jul 25 01:20:27.796679 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:27.796638    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:20:27.797133 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:27.796872    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:20:30.900971 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:30.900932    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:20:42.764951 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:42.764916    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"]
Jul 25 01:20:42.765442 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:42.765301    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb" podUID="2e82012a-df95-4d95-b57e-565e062c04db" containerName="payload-processing" containerID="cri-o://55d9bfc44dd8e8c60e002027fcfd8c95b2b69512de89e2c887adc8930d61196c" gracePeriod=30
Jul 25 01:20:42.776483 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:42.776454    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"]
Jul 25 01:20:42.776695 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:42.776669    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq" podUID="ed633317-af31-4e6d-9019-e4f62180c193" containerName="payload-pre-processing" containerID="cri-o://0d54da1f9b16d02ccbe6fce7aa62b332dbb01954ba9f74041393812a6641d315" gracePeriod=30
Jul 25 01:20:42.909142 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:42.909109    2566 generic.go:356] "Generic (PLEG): container finished" podID="2e82012a-df95-4d95-b57e-565e062c04db" containerID="55d9bfc44dd8e8c60e002027fcfd8c95b2b69512de89e2c887adc8930d61196c" exitCode=0
Jul 25 01:20:42.909276 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:42.909179    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb" event={"ID":"2e82012a-df95-4d95-b57e-565e062c04db","Type":"ContainerDied","Data":"55d9bfc44dd8e8c60e002027fcfd8c95b2b69512de89e2c887adc8930d61196c"}
Jul 25 01:20:42.910910 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:42.910884    2566 generic.go:356] "Generic (PLEG): container finished" podID="ed633317-af31-4e6d-9019-e4f62180c193" containerID="0d54da1f9b16d02ccbe6fce7aa62b332dbb01954ba9f74041393812a6641d315" exitCode=0
Jul 25 01:20:42.911011 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:42.910930    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq" event={"ID":"ed633317-af31-4e6d-9019-e4f62180c193","Type":"ContainerDied","Data":"0d54da1f9b16d02ccbe6fce7aa62b332dbb01954ba9f74041393812a6641d315"}
Jul 25 01:20:43.028892 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.028872    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:43.037823 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.037805    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:43.135360 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.135333    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vp2dx\" (UniqueName: \"kubernetes.io/projected/ed633317-af31-4e6d-9019-e4f62180c193-kube-api-access-vp2dx\") pod \"ed633317-af31-4e6d-9019-e4f62180c193\" (UID: \"ed633317-af31-4e6d-9019-e4f62180c193\") "
Jul 25 01:20:43.135488 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.135368    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljjrv\" (UniqueName: \"kubernetes.io/projected/2e82012a-df95-4d95-b57e-565e062c04db-kube-api-access-ljjrv\") pod \"2e82012a-df95-4d95-b57e-565e062c04db\" (UID: \"2e82012a-df95-4d95-b57e-565e062c04db\") "
Jul 25 01:20:43.135488 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.135394    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e82012a-df95-4d95-b57e-565e062c04db-plugins-config-volume\") pod \"2e82012a-df95-4d95-b57e-565e062c04db\" (UID: \"2e82012a-df95-4d95-b57e-565e062c04db\") "
Jul 25 01:20:43.135488 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.135417    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed633317-af31-4e6d-9019-e4f62180c193-plugins-config-volume\") pod \"ed633317-af31-4e6d-9019-e4f62180c193\" (UID: \"ed633317-af31-4e6d-9019-e4f62180c193\") "
Jul 25 01:20:43.135790 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.135762    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e82012a-df95-4d95-b57e-565e062c04db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e82012a-df95-4d95-b57e-565e062c04db" (UID: "2e82012a-df95-4d95-b57e-565e062c04db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:20:43.135891 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.135764    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed633317-af31-4e6d-9019-e4f62180c193-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed633317-af31-4e6d-9019-e4f62180c193" (UID: "ed633317-af31-4e6d-9019-e4f62180c193"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:20:43.137362 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.137336    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e82012a-df95-4d95-b57e-565e062c04db-kube-api-access-ljjrv" (OuterVolumeSpecName: "kube-api-access-ljjrv") pod "2e82012a-df95-4d95-b57e-565e062c04db" (UID: "2e82012a-df95-4d95-b57e-565e062c04db"). InnerVolumeSpecName "kube-api-access-ljjrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:43.137428 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.137382    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed633317-af31-4e6d-9019-e4f62180c193-kube-api-access-vp2dx" (OuterVolumeSpecName: "kube-api-access-vp2dx") pod "ed633317-af31-4e6d-9019-e4f62180c193" (UID: "ed633317-af31-4e6d-9019-e4f62180c193"). InnerVolumeSpecName "kube-api-access-vp2dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:43.236603 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.236581    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vp2dx\" (UniqueName: \"kubernetes.io/projected/ed633317-af31-4e6d-9019-e4f62180c193-kube-api-access-vp2dx\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:20:43.236603 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.236600    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljjrv\" (UniqueName: \"kubernetes.io/projected/2e82012a-df95-4d95-b57e-565e062c04db-kube-api-access-ljjrv\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:20:43.236722 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.236610    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e82012a-df95-4d95-b57e-565e062c04db-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:20:43.236722 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.236623    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed633317-af31-4e6d-9019-e4f62180c193-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:20:43.916982 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.916947    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"
Jul 25 01:20:43.917344 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.916951    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb" event={"ID":"2e82012a-df95-4d95-b57e-565e062c04db","Type":"ContainerDied","Data":"b9cf6ca90671ba65d190c08fd9adbc7554d252f645afd7b40ad12ba74aa3b592"}
Jul 25 01:20:43.917344 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.917137    2566 scope.go:117] "RemoveContainer" containerID="55d9bfc44dd8e8c60e002027fcfd8c95b2b69512de89e2c887adc8930d61196c"
Jul 25 01:20:43.918743 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.918719    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq" event={"ID":"ed633317-af31-4e6d-9019-e4f62180c193","Type":"ContainerDied","Data":"990530633ab5acaa39404d6a9b0e29c4a1fb6019d5d175004db6c025740c858c"}
Jul 25 01:20:43.918819 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.918796    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"
Jul 25 01:20:43.926515 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.926492    2566 scope.go:117] "RemoveContainer" containerID="0d54da1f9b16d02ccbe6fce7aa62b332dbb01954ba9f74041393812a6641d315"
Jul 25 01:20:43.942362 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.942338    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"]
Jul 25 01:20:43.945796 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.945748    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c59e95-dcbc59fb-l9bsq"]
Jul 25 01:20:43.955501 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.955475    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"]
Jul 25 01:20:43.960402 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:43.960379    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c59e95-859ffb87d8-bmkgb"]
Jul 25 01:20:44.171808 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:44.171782    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e82012a-df95-4d95-b57e-565e062c04db" path="/var/lib/kubelet/pods/2e82012a-df95-4d95-b57e-565e062c04db/volumes"
Jul 25 01:20:44.172162 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:44.172147    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed633317-af31-4e6d-9019-e4f62180c193" path="/var/lib/kubelet/pods/ed633317-af31-4e6d-9019-e4f62180c193/volumes"
Jul 25 01:20:46.392582 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:20:46.392543    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:21:09.983944 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:09.983854    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"]
Jul 25 01:21:09.984484 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:09.984131    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5" podUID="a3cb5e87-963b-44b3-bdcc-f51d93365c2d" containerName="payload-processing" containerID="cri-o://26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c" gracePeriod=30
Jul 25 01:21:09.990946 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:09.990915    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"]
Jul 25 01:21:09.991506 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:09.991167    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj" podUID="99a911a0-18a9-42c8-8046-89c849686124" containerName="payload-pre-processing" containerID="cri-o://a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8" gracePeriod=30
Jul 25 01:21:10.277054 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.277025    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:21:10.280080 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.280049    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:21:10.432639 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.432614    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99a911a0-18a9-42c8-8046-89c849686124-plugins-config-volume\") pod \"99a911a0-18a9-42c8-8046-89c849686124\" (UID: \"99a911a0-18a9-42c8-8046-89c849686124\") "
Jul 25 01:21:10.432771 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.432689    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-plugins-config-volume\") pod \"a3cb5e87-963b-44b3-bdcc-f51d93365c2d\" (UID: \"a3cb5e87-963b-44b3-bdcc-f51d93365c2d\") "
Jul 25 01:21:10.432861 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.432840    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhxpz\" (UniqueName: \"kubernetes.io/projected/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-kube-api-access-nhxpz\") pod \"a3cb5e87-963b-44b3-bdcc-f51d93365c2d\" (UID: \"a3cb5e87-963b-44b3-bdcc-f51d93365c2d\") "
Jul 25 01:21:10.432924 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.432894    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmntb\" (UniqueName: \"kubernetes.io/projected/99a911a0-18a9-42c8-8046-89c849686124-kube-api-access-lmntb\") pod \"99a911a0-18a9-42c8-8046-89c849686124\" (UID: \"99a911a0-18a9-42c8-8046-89c849686124\") "
Jul 25 01:21:10.432995 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.432974    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3cb5e87-963b-44b3-bdcc-f51d93365c2d" (UID: "a3cb5e87-963b-44b3-bdcc-f51d93365c2d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:21:10.433141 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.433042    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a911a0-18a9-42c8-8046-89c849686124-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "99a911a0-18a9-42c8-8046-89c849686124" (UID: "99a911a0-18a9-42c8-8046-89c849686124"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:21:10.433210 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.433162    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99a911a0-18a9-42c8-8046-89c849686124-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:21:10.433210 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.433184    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:21:10.434857 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.434833    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-kube-api-access-nhxpz" (OuterVolumeSpecName: "kube-api-access-nhxpz") pod "a3cb5e87-963b-44b3-bdcc-f51d93365c2d" (UID: "a3cb5e87-963b-44b3-bdcc-f51d93365c2d"). InnerVolumeSpecName "kube-api-access-nhxpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:21:10.434954 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.434914    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a911a0-18a9-42c8-8046-89c849686124-kube-api-access-lmntb" (OuterVolumeSpecName: "kube-api-access-lmntb") pod "99a911a0-18a9-42c8-8046-89c849686124" (UID: "99a911a0-18a9-42c8-8046-89c849686124"). InnerVolumeSpecName "kube-api-access-lmntb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:21:10.533890 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.533825    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmntb\" (UniqueName: \"kubernetes.io/projected/99a911a0-18a9-42c8-8046-89c849686124-kube-api-access-lmntb\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:21:10.533890 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:10.533852    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhxpz\" (UniqueName: \"kubernetes.io/projected/a3cb5e87-963b-44b3-bdcc-f51d93365c2d-kube-api-access-nhxpz\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:21:11.042571 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.042537    2566 generic.go:356] "Generic (PLEG): container finished" podID="a3cb5e87-963b-44b3-bdcc-f51d93365c2d" containerID="26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c" exitCode=0
Jul 25 01:21:11.042924 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.042593    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"
Jul 25 01:21:11.042924 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.042631    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5" event={"ID":"a3cb5e87-963b-44b3-bdcc-f51d93365c2d","Type":"ContainerDied","Data":"26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c"}
Jul 25 01:21:11.042924 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.042690    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5" event={"ID":"a3cb5e87-963b-44b3-bdcc-f51d93365c2d","Type":"ContainerDied","Data":"89261406a924370f4a2c95ec6b91f993a132d55ccabf931d18a45d3d9c547822"}
Jul 25 01:21:11.042924 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.042714    2566 scope.go:117] "RemoveContainer" containerID="26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c"
Jul 25 01:21:11.044195 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.044170    2566 generic.go:356] "Generic (PLEG): container finished" podID="99a911a0-18a9-42c8-8046-89c849686124" containerID="a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8" exitCode=0
Jul 25 01:21:11.044269 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.044233    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj" event={"ID":"99a911a0-18a9-42c8-8046-89c849686124","Type":"ContainerDied","Data":"a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8"}
Jul 25 01:21:11.044338 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.044275    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj" event={"ID":"99a911a0-18a9-42c8-8046-89c849686124","Type":"ContainerDied","Data":"f82578453f617d61c876e49b975fa222d56b39fa48599319caf46775a75443fe"}
Jul 25 01:21:11.044338 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.044234    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"
Jul 25 01:21:11.052562 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.052417    2566 scope.go:117] "RemoveContainer" containerID="26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c"
Jul 25 01:21:11.052744 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:21:11.052713    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c\": container with ID starting with 26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c not found: ID does not exist" containerID="26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c"
Jul 25 01:21:11.052823 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.052754    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c"} err="failed to get container status \"26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c\": rpc error: code = NotFound desc = could not find container \"26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c\": container with ID starting with 26c2adb6ccdddd488279a18f231f64fc4ced41976fad998e34706e87cb2eb88c not found: ID does not exist"
Jul 25 01:21:11.052823 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.052779    2566 scope.go:117] "RemoveContainer" containerID="a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8"
Jul 25 01:21:11.060849 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.060830    2566 scope.go:117] "RemoveContainer" containerID="a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8"
Jul 25 01:21:11.061166 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:21:11.061141    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8\": container with ID starting with a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8 not found: ID does not exist" containerID="a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8"
Jul 25 01:21:11.061267 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.061169    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8"} err="failed to get container status \"a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8\": rpc error: code = NotFound desc = could not find container \"a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8\": container with ID starting with a6dc7331078b2c96943cab7ad89e2d449b9ac7183fb72992e32d7f74e2f6e3c8 not found: ID does not exist"
Jul 25 01:21:11.066013 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.065989    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"]
Jul 25 01:21:11.071590 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.071567    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-015d51-865f796996-5wvjj"]
Jul 25 01:21:11.080306 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.080283    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"]
Jul 25 01:21:11.082311 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:11.082289    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-015d51-7cc8cb5fc4-gpzr5"]
Jul 25 01:21:12.170957 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:12.170916    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99a911a0-18a9-42c8-8046-89c849686124" path="/var/lib/kubelet/pods/99a911a0-18a9-42c8-8046-89c849686124/volumes"
Jul 25 01:21:12.171387 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:12.171322    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3cb5e87-963b-44b3-bdcc-f51d93365c2d" path="/var/lib/kubelet/pods/a3cb5e87-963b-44b3-bdcc-f51d93365c2d/volumes"
Jul 25 01:21:14.589510 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:14.589472    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:21:18.314791 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:18.314763    2566 scope.go:117] "RemoveContainer" containerID="c5a3a15a2d1ce0324e032b4326be20a21e1d8855f3cb39a57321cc6a77a4478a"
Jul 25 01:21:25.854869 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.854837    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"]
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855128    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99a911a0-18a9-42c8-8046-89c849686124" containerName="payload-pre-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855140    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="99a911a0-18a9-42c8-8046-89c849686124" containerName="payload-pre-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855153    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e82012a-df95-4d95-b57e-565e062c04db" containerName="payload-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855159    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e82012a-df95-4d95-b57e-565e062c04db" containerName="payload-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855166    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3cb5e87-963b-44b3-bdcc-f51d93365c2d" containerName="payload-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855172    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3cb5e87-963b-44b3-bdcc-f51d93365c2d" containerName="payload-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855184    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed633317-af31-4e6d-9019-e4f62180c193" containerName="payload-pre-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855190    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed633317-af31-4e6d-9019-e4f62180c193" containerName="payload-pre-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855237    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3cb5e87-963b-44b3-bdcc-f51d93365c2d" containerName="payload-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855244    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e82012a-df95-4d95-b57e-565e062c04db" containerName="payload-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855250    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed633317-af31-4e6d-9019-e4f62180c193" containerName="payload-pre-processing"
Jul 25 01:21:25.855331 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.855256    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="99a911a0-18a9-42c8-8046-89c849686124" containerName="payload-pre-processing"
Jul 25 01:21:25.858098 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.858042    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:21:25.858758 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.858736    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"]
Jul 25 01:21:25.860230 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.860207    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-a1fe77\""
Jul 25 01:21:25.861587 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.861565    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:21:25.863654 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.863634    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-a1fe77-dockercfg-8jgqs\""
Jul 25 01:21:25.867030 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.867011    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"]
Jul 25 01:21:25.871840 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.871817    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"]
Jul 25 01:21:25.944383 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.944357    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h\" (UID: \"9252a56a-fec6-4100-b0c4-bdafa9a0ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:21:25.944472 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.944395    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldj5b\" (UniqueName: \"kubernetes.io/projected/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-kube-api-access-ldj5b\") pod \"payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h\" (UID: \"9252a56a-fec6-4100-b0c4-bdafa9a0ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:21:25.944472 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.944431    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5vw5\" (UniqueName: \"kubernetes.io/projected/292eeca0-d800-4fb0-9d3b-6aa4763b736e-kube-api-access-z5vw5\") pod \"payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2\" (UID: \"292eeca0-d800-4fb0-9d3b-6aa4763b736e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:21:25.944472 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:25.944469    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/292eeca0-d800-4fb0-9d3b-6aa4763b736e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2\" (UID: \"292eeca0-d800-4fb0-9d3b-6aa4763b736e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:21:26.044802 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.044778    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/292eeca0-d800-4fb0-9d3b-6aa4763b736e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2\" (UID: \"292eeca0-d800-4fb0-9d3b-6aa4763b736e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:21:26.044925 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.044819    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h\" (UID: \"9252a56a-fec6-4100-b0c4-bdafa9a0ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:21:26.044925 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.044852    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldj5b\" (UniqueName: \"kubernetes.io/projected/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-kube-api-access-ldj5b\") pod \"payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h\" (UID: \"9252a56a-fec6-4100-b0c4-bdafa9a0ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:21:26.044925 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.044875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5vw5\" (UniqueName: \"kubernetes.io/projected/292eeca0-d800-4fb0-9d3b-6aa4763b736e-kube-api-access-z5vw5\") pod \"payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2\" (UID: \"292eeca0-d800-4fb0-9d3b-6aa4763b736e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:21:26.045508 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.045486    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h\" (UID: \"9252a56a-fec6-4100-b0c4-bdafa9a0ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:21:26.045598 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.045511    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/292eeca0-d800-4fb0-9d3b-6aa4763b736e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2\" (UID: \"292eeca0-d800-4fb0-9d3b-6aa4763b736e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:21:26.052245 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.052221    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldj5b\" (UniqueName: \"kubernetes.io/projected/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-kube-api-access-ldj5b\") pod \"payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h\" (UID: \"9252a56a-fec6-4100-b0c4-bdafa9a0ab29\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:21:26.052324 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.052289    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5vw5\" (UniqueName: \"kubernetes.io/projected/292eeca0-d800-4fb0-9d3b-6aa4763b736e-kube-api-access-z5vw5\") pod \"payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2\" (UID: \"292eeca0-d800-4fb0-9d3b-6aa4763b736e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:21:26.169403 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.169382    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:21:26.176147 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.176130    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:21:26.300943 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.300918    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"]
Jul 25 01:21:26.303619 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:21:26.303588    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9252a56a_fec6_4100_b0c4_bdafa9a0ab29.slice/crio-fe90723410cc05314cddf0f4d6e0a1147013db87123b8967487f66b691a86213 WatchSource:0}: Error finding container fe90723410cc05314cddf0f4d6e0a1147013db87123b8967487f66b691a86213: Status 404 returned error can't find the container with id fe90723410cc05314cddf0f4d6e0a1147013db87123b8967487f66b691a86213
Jul 25 01:21:26.320360 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:26.320338    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"]
Jul 25 01:21:26.325440 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:21:26.325416    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod292eeca0_d800_4fb0_9d3b_6aa4763b736e.slice/crio-9eede4a07a07df449853c87a98a25876eb1f5d7a73bf27b80e8b118f9812b322 WatchSource:0}: Error finding container 9eede4a07a07df449853c87a98a25876eb1f5d7a73bf27b80e8b118f9812b322: Status 404 returned error can't find the container with id 9eede4a07a07df449853c87a98a25876eb1f5d7a73bf27b80e8b118f9812b322
Jul 25 01:21:27.116544 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:27.116511    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h" event={"ID":"9252a56a-fec6-4100-b0c4-bdafa9a0ab29","Type":"ContainerStarted","Data":"e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800"}
Jul 25 01:21:27.116970 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:27.116574    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:21:27.116970 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:27.116601    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h" event={"ID":"9252a56a-fec6-4100-b0c4-bdafa9a0ab29","Type":"ContainerStarted","Data":"fe90723410cc05314cddf0f4d6e0a1147013db87123b8967487f66b691a86213"}
Jul 25 01:21:27.118021 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:27.117996    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2" event={"ID":"292eeca0-d800-4fb0-9d3b-6aa4763b736e","Type":"ContainerStarted","Data":"dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04"}
Jul 25 01:21:27.118147 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:27.118035    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2" event={"ID":"292eeca0-d800-4fb0-9d3b-6aa4763b736e","Type":"ContainerStarted","Data":"9eede4a07a07df449853c87a98a25876eb1f5d7a73bf27b80e8b118f9812b322"}
Jul 25 01:21:27.118147 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:27.118099    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:21:27.130902 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:27.130866    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h" podStartSLOduration=2.130854438 podStartE2EDuration="2.130854438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:21:27.130207623 +0000 UTC m=+2709.518730027" watchObservedRunningTime="2026-07-25 01:21:27.130854438 +0000 UTC m=+2709.519376841"
Jul 25 01:21:27.144517 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:27.144475    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2" podStartSLOduration=2.144459396 podStartE2EDuration="2.144459396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:21:27.144058416 +0000 UTC m=+2709.532580819" watchObservedRunningTime="2026-07-25 01:21:27.144459396 +0000 UTC m=+2709.532981800"
Jul 25 01:21:32.783407 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:32.783376    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:21:34.055850 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.055812    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"]
Jul 25 01:21:34.059385 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.059366    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:21:34.061684 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.061659    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-d50ef4-dockercfg-7nkqf\""
Jul 25 01:21:34.061821 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.061694    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-d50ef4\""
Jul 25 01:21:34.069380 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.069353    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"]
Jul 25 01:21:34.083104 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.083050    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"]
Jul 25 01:21:34.086556 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.086538    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:21:34.094502 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.094479    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"]
Jul 25 01:21:34.105036 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.105014    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm\" (UID: \"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:21:34.105139 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.105042    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cnx5\" (UniqueName: \"kubernetes.io/projected/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-kube-api-access-2cnx5\") pod \"payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm\" (UID: \"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:21:34.105139 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.105057    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92e8bc11-65b6-4005-bcd4-ee3ab005745b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c\" (UID: \"92e8bc11-65b6-4005-bcd4-ee3ab005745b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:21:34.105139 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.105098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmkt2\" (UniqueName: \"kubernetes.io/projected/92e8bc11-65b6-4005-bcd4-ee3ab005745b-kube-api-access-rmkt2\") pod \"payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c\" (UID: \"92e8bc11-65b6-4005-bcd4-ee3ab005745b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:21:34.205460 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.205430    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm\" (UID: \"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:21:34.205460 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.205460    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cnx5\" (UniqueName: \"kubernetes.io/projected/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-kube-api-access-2cnx5\") pod \"payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm\" (UID: \"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:21:34.205625 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.205479    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92e8bc11-65b6-4005-bcd4-ee3ab005745b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c\" (UID: \"92e8bc11-65b6-4005-bcd4-ee3ab005745b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:21:34.205625 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.205496    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmkt2\" (UniqueName: \"kubernetes.io/projected/92e8bc11-65b6-4005-bcd4-ee3ab005745b-kube-api-access-rmkt2\") pod \"payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c\" (UID: \"92e8bc11-65b6-4005-bcd4-ee3ab005745b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:21:34.206360 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.206338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92e8bc11-65b6-4005-bcd4-ee3ab005745b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c\" (UID: \"92e8bc11-65b6-4005-bcd4-ee3ab005745b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:21:34.206404 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.206354    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm\" (UID: \"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:21:34.213273 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.213246    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmkt2\" (UniqueName: \"kubernetes.io/projected/92e8bc11-65b6-4005-bcd4-ee3ab005745b-kube-api-access-rmkt2\") pod \"payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c\" (UID: \"92e8bc11-65b6-4005-bcd4-ee3ab005745b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:21:34.213484 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.213461    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cnx5\" (UniqueName: \"kubernetes.io/projected/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-kube-api-access-2cnx5\") pod \"payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm\" (UID: \"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:21:34.370431 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.370408    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:21:34.396446 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.396424    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:21:34.511080 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.511034    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"]
Jul 25 01:21:34.513381 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:21:34.513357    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4bded37_5da4_4e0a_81c6_a4fa62f9b8a2.slice/crio-3f9e2f5038c5fba1131400ec4cb9f94947227bd496d14e200e901fe78f16fb61 WatchSource:0}: Error finding container 3f9e2f5038c5fba1131400ec4cb9f94947227bd496d14e200e901fe78f16fb61: Status 404 returned error can't find the container with id 3f9e2f5038c5fba1131400ec4cb9f94947227bd496d14e200e901fe78f16fb61
Jul 25 01:21:34.734730 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:34.734674    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"]
Jul 25 01:21:34.736598 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:21:34.736570    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e8bc11_65b6_4005_bcd4_ee3ab005745b.slice/crio-ac95385a2f9dd9c20438e099b8412ff4d98802d1e9cd771e74f8468c96109549 WatchSource:0}: Error finding container ac95385a2f9dd9c20438e099b8412ff4d98802d1e9cd771e74f8468c96109549: Status 404 returned error can't find the container with id ac95385a2f9dd9c20438e099b8412ff4d98802d1e9cd771e74f8468c96109549
Jul 25 01:21:35.158407 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:35.158372    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm" event={"ID":"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2","Type":"ContainerStarted","Data":"2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff"}
Jul 25 01:21:35.158829 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:35.158534    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:21:35.158829 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:35.158569    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm" event={"ID":"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2","Type":"ContainerStarted","Data":"3f9e2f5038c5fba1131400ec4cb9f94947227bd496d14e200e901fe78f16fb61"}
Jul 25 01:21:35.159899 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:35.159878    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c" event={"ID":"92e8bc11-65b6-4005-bcd4-ee3ab005745b","Type":"ContainerStarted","Data":"d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472"}
Jul 25 01:21:35.159987 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:35.159924    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c" event={"ID":"92e8bc11-65b6-4005-bcd4-ee3ab005745b","Type":"ContainerStarted","Data":"ac95385a2f9dd9c20438e099b8412ff4d98802d1e9cd771e74f8468c96109549"}
Jul 25 01:21:35.160021 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:35.159996    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:21:35.174329 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:35.174283    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm" podStartSLOduration=1.174270637 podStartE2EDuration="1.174270637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:21:35.172788467 +0000 UTC m=+2717.561310884" watchObservedRunningTime="2026-07-25 01:21:35.174270637 +0000 UTC m=+2717.562793039"
Jul 25 01:21:35.187346 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:35.187308    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c" podStartSLOduration=1.187295968 podStartE2EDuration="1.187295968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:21:35.186370344 +0000 UTC m=+2717.574892746" watchObservedRunningTime="2026-07-25 01:21:35.187295968 +0000 UTC m=+2717.575818371"
Jul 25 01:21:38.123877 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:38.123835    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:21:38.124347 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:38.123936    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:21:46.172182 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:46.172148    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:21:46.172644 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:46.172215    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:21:47.190937 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:47.190896    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:21:55.292085 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:21:55.292017    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:22:18.346526 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:18.346496    2566 scope.go:117] "RemoveContainer" containerID="1f054965cd2dcf012a045576e7998ea830e5beed022de32c7cbece9cc83fc144"
Jul 25 01:22:25.493789 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.493744    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"]
Jul 25 01:22:25.498898 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.498875    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.501163 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.501130    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d50ef4\"/\"openshift-service-ca.crt\""
Jul 25 01:22:25.502170 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.502143    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d50ef4\"/\"kube-root-ca.crt\""
Jul 25 01:22:25.502725 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.502697    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d50ef4\"/\"default-dockercfg-zmhz4\""
Jul 25 01:22:25.502872 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.502847    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d50ef4\"/\"auth-test-model-d50ef4-kserve-self-signed-certs\""
Jul 25 01:22:25.505222 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.505195    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"]
Jul 25 01:22:25.574898 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.574873    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-dshm\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.575021 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.574911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-home\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.575021 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.574937    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-tmp-dir\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.575118 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.575015    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swwh4\" (UniqueName: \"kubernetes.io/projected/a7274417-7d25-4d88-910d-d6f3fa0563a0-kube-api-access-swwh4\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.575118 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.575082    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-model-cache\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.575118 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.575100    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7274417-7d25-4d88-910d-d6f3fa0563a0-tls-certs\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.675872 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.675844    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swwh4\" (UniqueName: \"kubernetes.io/projected/a7274417-7d25-4d88-910d-d6f3fa0563a0-kube-api-access-swwh4\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.675979 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.675887    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-model-cache\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.675979 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.675904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7274417-7d25-4d88-910d-d6f3fa0563a0-tls-certs\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.675979 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.675929    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-dshm\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.675979 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.675969    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-home\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.676152 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.676011    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-tmp-dir\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.676328 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.676307    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-model-cache\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.676397 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.676379    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-home\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.676466 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.676448    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-tmp-dir\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.678353 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.678332    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-dshm\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.678492 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.678474    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7274417-7d25-4d88-910d-d6f3fa0563a0-tls-certs\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.683699 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.683675    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swwh4\" (UniqueName: \"kubernetes.io/projected/a7274417-7d25-4d88-910d-d6f3fa0563a0-kube-api-access-swwh4\") pod \"auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") " pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.810111 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.810030    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:25.935001 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:25.934976    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"]
Jul 25 01:22:25.937195 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:22:25.937168    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7274417_7d25_4d88_910d_d6f3fa0563a0.slice/crio-05146dc00627d667df0999b1b9b15940af98ce8e9aa45f049fd597c8996834b8 WatchSource:0}: Error finding container 05146dc00627d667df0999b1b9b15940af98ce8e9aa45f049fd597c8996834b8: Status 404 returned error can't find the container with id 05146dc00627d667df0999b1b9b15940af98ce8e9aa45f049fd597c8996834b8
Jul 25 01:22:26.389198 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:26.389161    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw" event={"ID":"a7274417-7d25-4d88-910d-d6f3fa0563a0","Type":"ContainerStarted","Data":"30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65"}
Jul 25 01:22:26.389198 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:26.389200    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw" event={"ID":"a7274417-7d25-4d88-910d-d6f3fa0563a0","Type":"ContainerStarted","Data":"05146dc00627d667df0999b1b9b15940af98ce8e9aa45f049fd597c8996834b8"}
Jul 25 01:22:26.404751 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:26.404702    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw" podStartSLOduration=1.404687957 podStartE2EDuration="1.404687957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:22:26.403558711 +0000 UTC m=+2768.792081140" watchObservedRunningTime="2026-07-25 01:22:26.404687957 +0000 UTC m=+2768.793210359"
Jul 25 01:22:29.081022 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:29.080977    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:22:35.810378 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:35.810341    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:35.810782 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:35.810396    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:35.822317 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:35.822284    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:36.443367 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:36.443341    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:22:53.993455 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:22:53.993413    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:23:20.190788 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:23:20.190751    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:23:36.985549 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:23:36.985507    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:23:59.080841 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:23:59.080799    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:24:17.189851 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:24:17.189808    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:24:40.084921 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:24:40.084874    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:24:57.815217 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:24:57.815175    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:25:20.684369 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:25:20.684325    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:25:38.283241 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:25:38.283204    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:26:03.591445 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:03.591344    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:26:18.385387 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:18.385358    2566 scope.go:117] "RemoveContainer" containerID="f02b3a0e63993ba7176bb6bc514dfd1e83ce71eef2def8444b7c1adabd344751"
Jul 25 01:26:21.602013 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:21.601972    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:26:43.385527 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:43.385485    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:26:49.712210 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:49.712168    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"]
Jul 25 01:26:58.588838 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.588799    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"]
Jul 25 01:26:58.592142 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.592119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.594411 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.594375    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a1fe77\"/\"openshift-service-ca.crt\""
Jul 25 01:26:58.594512 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.594375    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a1fe77\"/\"default-dockercfg-58sdk\""
Jul 25 01:26:58.595498 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.595481    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a1fe77\"/\"sub-test-model-a1fe77-kserve-self-signed-certs\""
Jul 25 01:26:58.595580 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.595524    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a1fe77\"/\"kube-root-ca.crt\""
Jul 25 01:26:58.601950 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.601924    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"]
Jul 25 01:26:58.648590 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.648561    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-dshm\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.648700 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.648607    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm85r\" (UniqueName: \"kubernetes.io/projected/07f9a885-30a2-47a0-9bf4-6947cd849807-kube-api-access-zm85r\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.648700 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.648640    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-tmp-dir\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.648700 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.648680    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-home\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.648816 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.648759    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-model-cache\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.648816 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.648791    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07f9a885-30a2-47a0-9bf4-6947cd849807-tls-certs\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.749969 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.749946    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-model-cache\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.750164 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.749976    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07f9a885-30a2-47a0-9bf4-6947cd849807-tls-certs\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.750164 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.749997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-dshm\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.750164 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.750021    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm85r\" (UniqueName: \"kubernetes.io/projected/07f9a885-30a2-47a0-9bf4-6947cd849807-kube-api-access-zm85r\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.750164 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.750040    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-tmp-dir\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.750526 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.750161    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-home\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.750526 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.750394    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-model-cache\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.750526 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.750455    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-tmp-dir\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.750526 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.750500    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-home\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.752306 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.752279    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-dshm\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.752601 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.752580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07f9a885-30a2-47a0-9bf4-6947cd849807-tls-certs\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.757503 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.757477    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm85r\" (UniqueName: \"kubernetes.io/projected/07f9a885-30a2-47a0-9bf4-6947cd849807-kube-api-access-zm85r\") pod \"sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") " pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:58.902763 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:58.902732    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:26:59.024693 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:59.024663    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"]
Jul 25 01:26:59.027875 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:26:59.027849    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07f9a885_30a2_47a0_9bf4_6947cd849807.slice/crio-dd8db979f622b6dc68d325c97b23dc5a5df964cb09b2a729aea1844ce6a2f872 WatchSource:0}: Error finding container dd8db979f622b6dc68d325c97b23dc5a5df964cb09b2a729aea1844ce6a2f872: Status 404 returned error can't find the container with id dd8db979f622b6dc68d325c97b23dc5a5df964cb09b2a729aea1844ce6a2f872
Jul 25 01:26:59.621105 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:59.621050    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q" event={"ID":"07f9a885-30a2-47a0-9bf4-6947cd849807","Type":"ContainerStarted","Data":"fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f"}
Jul 25 01:26:59.621479 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:59.621118    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q" event={"ID":"07f9a885-30a2-47a0-9bf4-6947cd849807","Type":"ContainerStarted","Data":"dd8db979f622b6dc68d325c97b23dc5a5df964cb09b2a729aea1844ce6a2f872"}
Jul 25 01:26:59.638615 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:26:59.638561    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q" podStartSLOduration=1.6385452219999999 podStartE2EDuration="1.638545222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:26:59.637467571 +0000 UTC m=+3042.025989970" watchObservedRunningTime="2026-07-25 01:26:59.638545222 +0000 UTC m=+3042.027067699"
Jul 25 01:27:01.390312 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:01.390271    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:27:04.731372 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:04.731316    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw" podUID="a7274417-7d25-4d88-910d-d6f3fa0563a0" containerName="main" containerID="cri-o://30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65" gracePeriod=45
Jul 25 01:27:04.984544 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:04.984492    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:27:05.098006 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.097979    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-home\") pod \"a7274417-7d25-4d88-910d-d6f3fa0563a0\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") "
Jul 25 01:27:05.098147 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.098030    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-dshm\") pod \"a7274417-7d25-4d88-910d-d6f3fa0563a0\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") "
Jul 25 01:27:05.098147 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.098059    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-tmp-dir\") pod \"a7274417-7d25-4d88-910d-d6f3fa0563a0\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") "
Jul 25 01:27:05.098147 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.098114    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7274417-7d25-4d88-910d-d6f3fa0563a0-tls-certs\") pod \"a7274417-7d25-4d88-910d-d6f3fa0563a0\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") "
Jul 25 01:27:05.098281 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.098148    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-model-cache\") pod \"a7274417-7d25-4d88-910d-d6f3fa0563a0\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") "
Jul 25 01:27:05.098281 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.098183    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swwh4\" (UniqueName: \"kubernetes.io/projected/a7274417-7d25-4d88-910d-d6f3fa0563a0-kube-api-access-swwh4\") pod \"a7274417-7d25-4d88-910d-d6f3fa0563a0\" (UID: \"a7274417-7d25-4d88-910d-d6f3fa0563a0\") "
Jul 25 01:27:05.098281 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.098256    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-home" (OuterVolumeSpecName: "home") pod "a7274417-7d25-4d88-910d-d6f3fa0563a0" (UID: "a7274417-7d25-4d88-910d-d6f3fa0563a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:27:05.098440 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.098351    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a7274417-7d25-4d88-910d-d6f3fa0563a0" (UID: "a7274417-7d25-4d88-910d-d6f3fa0563a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:27:05.098494 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.098448    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "a7274417-7d25-4d88-910d-d6f3fa0563a0" (UID: "a7274417-7d25-4d88-910d-d6f3fa0563a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:27:05.098529 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.098504    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:27:05.098529 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.098522    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:27:05.100341 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.100314    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7274417-7d25-4d88-910d-d6f3fa0563a0-kube-api-access-swwh4" (OuterVolumeSpecName: "kube-api-access-swwh4") pod "a7274417-7d25-4d88-910d-d6f3fa0563a0" (UID: "a7274417-7d25-4d88-910d-d6f3fa0563a0"). InnerVolumeSpecName "kube-api-access-swwh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:27:05.100341 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.100327    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7274417-7d25-4d88-910d-d6f3fa0563a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a7274417-7d25-4d88-910d-d6f3fa0563a0" (UID: "a7274417-7d25-4d88-910d-d6f3fa0563a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:27:05.100478 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.100321    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-dshm" (OuterVolumeSpecName: "dshm") pod "a7274417-7d25-4d88-910d-d6f3fa0563a0" (UID: "a7274417-7d25-4d88-910d-d6f3fa0563a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:27:05.199200 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.199173    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7274417-7d25-4d88-910d-d6f3fa0563a0-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:27:05.199200 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.199198    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:27:05.199200 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.199207    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swwh4\" (UniqueName: \"kubernetes.io/projected/a7274417-7d25-4d88-910d-d6f3fa0563a0-kube-api-access-swwh4\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:27:05.199365 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.199215    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7274417-7d25-4d88-910d-d6f3fa0563a0-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:27:05.650423 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.650395    2566 generic.go:356] "Generic (PLEG): container finished" podID="a7274417-7d25-4d88-910d-d6f3fa0563a0" containerID="30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65" exitCode=0
Jul 25 01:27:05.650555 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.650430    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw" event={"ID":"a7274417-7d25-4d88-910d-d6f3fa0563a0","Type":"ContainerDied","Data":"30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65"}
Jul 25 01:27:05.650555 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.650482    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw" event={"ID":"a7274417-7d25-4d88-910d-d6f3fa0563a0","Type":"ContainerDied","Data":"05146dc00627d667df0999b1b9b15940af98ce8e9aa45f049fd597c8996834b8"}
Jul 25 01:27:05.650555 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.650501    2566 scope.go:117] "RemoveContainer" containerID="30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65"
Jul 25 01:27:05.650555 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.650447    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"
Jul 25 01:27:05.660435 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.660412    2566 scope.go:117] "RemoveContainer" containerID="30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65"
Jul 25 01:27:05.660728 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:27:05.660700    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65\": container with ID starting with 30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65 not found: ID does not exist" containerID="30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65"
Jul 25 01:27:05.660779 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.660742    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65"} err="failed to get container status \"30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65\": rpc error: code = NotFound desc = could not find container \"30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65\": container with ID starting with 30179e99c1479a874a4e50e2ef53be96dc784ae5aa276fa5b6c6b17a455c4d65 not found: ID does not exist"
Jul 25 01:27:05.671419 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.671394    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"]
Jul 25 01:27:05.677093 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:05.677049    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/auth-test-model-d50ef4-kserve-c4bd774c5-hgwtw"]
Jul 25 01:27:06.172424 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:06.172399    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7274417-7d25-4d88-910d-d6f3fa0563a0" path="/var/lib/kubelet/pods/a7274417-7d25-4d88-910d-d6f3fa0563a0/volumes"
Jul 25 01:27:08.903036 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:08.903000    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:27:08.903555 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:08.903050    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:27:08.915261 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:08.915229    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:27:09.680389 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:09.680314    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:27:24.907920 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:24.907880    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:27:57.887104 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:27:57.887047    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:28:18.388851 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:18.388815    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:28:24.569082 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:24.569033    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"]
Jul 25 01:28:39.185304 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.185263    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:28:39.588807 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.588687    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q" podUID="07f9a885-30a2-47a0-9bf4-6947cd849807" containerName="main" containerID="cri-o://fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f" gracePeriod=45
Jul 25 01:28:39.670453 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.670420    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q" podUID="07f9a885-30a2-47a0-9bf4-6947cd849807" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": dial tcp 10.134.0.47:8000: connect: connection refused"
Jul 25 01:28:39.836946 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.836924    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:28:39.883162 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.883143    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-model-cache\") pod \"07f9a885-30a2-47a0-9bf4-6947cd849807\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") "
Jul 25 01:28:39.883266 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.883171    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07f9a885-30a2-47a0-9bf4-6947cd849807-tls-certs\") pod \"07f9a885-30a2-47a0-9bf4-6947cd849807\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") "
Jul 25 01:28:39.883266 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.883192    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zm85r\" (UniqueName: \"kubernetes.io/projected/07f9a885-30a2-47a0-9bf4-6947cd849807-kube-api-access-zm85r\") pod \"07f9a885-30a2-47a0-9bf4-6947cd849807\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") "
Jul 25 01:28:39.883266 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.883210    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-home\") pod \"07f9a885-30a2-47a0-9bf4-6947cd849807\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") "
Jul 25 01:28:39.883266 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.883250    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-dshm\") pod \"07f9a885-30a2-47a0-9bf4-6947cd849807\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") "
Jul 25 01:28:39.883452 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.883284    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-tmp-dir\") pod \"07f9a885-30a2-47a0-9bf4-6947cd849807\" (UID: \"07f9a885-30a2-47a0-9bf4-6947cd849807\") "
Jul 25 01:28:39.883452 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.883394    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-model-cache" (OuterVolumeSpecName: "model-cache") pod "07f9a885-30a2-47a0-9bf4-6947cd849807" (UID: "07f9a885-30a2-47a0-9bf4-6947cd849807"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:28:39.883553 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.883453    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-home" (OuterVolumeSpecName: "home") pod "07f9a885-30a2-47a0-9bf4-6947cd849807" (UID: "07f9a885-30a2-47a0-9bf4-6947cd849807"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:28:39.883607 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.883594    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07f9a885-30a2-47a0-9bf4-6947cd849807" (UID: "07f9a885-30a2-47a0-9bf4-6947cd849807"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:28:39.885409 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.885381    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-dshm" (OuterVolumeSpecName: "dshm") pod "07f9a885-30a2-47a0-9bf4-6947cd849807" (UID: "07f9a885-30a2-47a0-9bf4-6947cd849807"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:28:39.885491 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.885413    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07f9a885-30a2-47a0-9bf4-6947cd849807-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07f9a885-30a2-47a0-9bf4-6947cd849807" (UID: "07f9a885-30a2-47a0-9bf4-6947cd849807"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:28:39.885491 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.885418    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f9a885-30a2-47a0-9bf4-6947cd849807-kube-api-access-zm85r" (OuterVolumeSpecName: "kube-api-access-zm85r") pod "07f9a885-30a2-47a0-9bf4-6947cd849807" (UID: "07f9a885-30a2-47a0-9bf4-6947cd849807"). InnerVolumeSpecName "kube-api-access-zm85r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:28:39.984168 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.984143    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:28:39.984168 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.984163    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:28:39.984168 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.984171    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07f9a885-30a2-47a0-9bf4-6947cd849807-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:28:39.984324 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.984179    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zm85r\" (UniqueName: \"kubernetes.io/projected/07f9a885-30a2-47a0-9bf4-6947cd849807-kube-api-access-zm85r\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:28:39.984324 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.984188    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:28:39.984324 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:39.984195    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07f9a885-30a2-47a0-9bf4-6947cd849807-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:28:40.069991 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:40.069958    2566 generic.go:356] "Generic (PLEG): container finished" podID="07f9a885-30a2-47a0-9bf4-6947cd849807" containerID="fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f" exitCode=0
Jul 25 01:28:40.070107 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:40.070002    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"
Jul 25 01:28:40.070107 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:40.070055    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q" event={"ID":"07f9a885-30a2-47a0-9bf4-6947cd849807","Type":"ContainerDied","Data":"fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f"}
Jul 25 01:28:40.070191 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:40.070131    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q" event={"ID":"07f9a885-30a2-47a0-9bf4-6947cd849807","Type":"ContainerDied","Data":"dd8db979f622b6dc68d325c97b23dc5a5df964cb09b2a729aea1844ce6a2f872"}
Jul 25 01:28:40.070191 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:40.070151    2566 scope.go:117] "RemoveContainer" containerID="fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f"
Jul 25 01:28:40.079418 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:40.079400    2566 scope.go:117] "RemoveContainer" containerID="fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f"
Jul 25 01:28:40.079729 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:28:40.079706    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f\": container with ID starting with fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f not found: ID does not exist" containerID="fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f"
Jul 25 01:28:40.079811 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:40.079742    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f"} err="failed to get container status \"fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f\": rpc error: code = NotFound desc = could not find container \"fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f\": container with ID starting with fc717c9040e11549730860d9d874535315e3901abf46559bba0a4a34f7928c1f not found: ID does not exist"
Jul 25 01:28:40.090359 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:40.090332    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"]
Jul 25 01:28:40.096170 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:40.096148    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a1fe77/sub-test-model-a1fe77-kserve-79cc5d7fd9-9cm2q"]
Jul 25 01:28:40.173847 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:40.173793    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07f9a885-30a2-47a0-9bf4-6947cd849807" path="/var/lib/kubelet/pods/07f9a885-30a2-47a0-9bf4-6947cd849807/volumes"
Jul 25 01:28:51.391412 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:28:51.391370    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:29:07.673045 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.672985    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"]
Jul 25 01:29:07.673619 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.673351    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7274417-7d25-4d88-910d-d6f3fa0563a0" containerName="main"
Jul 25 01:29:07.673619 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.673365    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7274417-7d25-4d88-910d-d6f3fa0563a0" containerName="main"
Jul 25 01:29:07.673619 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.673374    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f9a885-30a2-47a0-9bf4-6947cd849807" containerName="main"
Jul 25 01:29:07.673619 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.673380    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f9a885-30a2-47a0-9bf4-6947cd849807" containerName="main"
Jul 25 01:29:07.673619 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.673429    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7274417-7d25-4d88-910d-d6f3fa0563a0" containerName="main"
Jul 25 01:29:07.673619 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.673439    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f9a885-30a2-47a0-9bf4-6947cd849807" containerName="main"
Jul 25 01:29:07.676184 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.676161    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.679187 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.679149    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d50ef4\"/\"rate-test-model-d50ef4-kserve-self-signed-certs\""
Jul 25 01:29:07.679322 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.679149    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d50ef4\"/\"default-dockercfg-zmhz4\""
Jul 25 01:29:07.679322 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.679227    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d50ef4\"/\"kube-root-ca.crt\""
Jul 25 01:29:07.679322 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.679210    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d50ef4\"/\"openshift-service-ca.crt\""
Jul 25 01:29:07.685122 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.685090    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"]
Jul 25 01:29:07.785641 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.785613    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-dshm\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.785641 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.785645    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-home\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.785793 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.785670    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-model-cache\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.785793 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.785722    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-tmp-dir\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.785793 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.785746    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbl4z\" (UniqueName: \"kubernetes.io/projected/4bb0de17-a082-4c75-a783-4e715e0789ea-kube-api-access-sbl4z\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.785891 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.785820    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bb0de17-a082-4c75-a783-4e715e0789ea-tls-certs\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.886377 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.886347    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bb0de17-a082-4c75-a783-4e715e0789ea-tls-certs\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.886377 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.886378    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-dshm\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.886554 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.886397    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-home\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.886554 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.886423    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-model-cache\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.886554 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.886457    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-tmp-dir\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.886554 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.886481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbl4z\" (UniqueName: \"kubernetes.io/projected/4bb0de17-a082-4c75-a783-4e715e0789ea-kube-api-access-sbl4z\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.886868 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.886836    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-model-cache\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.886979 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.886866    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-home\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.886979 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.886915    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-tmp-dir\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.888609 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.888588    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-dshm\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.888863 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.888843    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bb0de17-a082-4c75-a783-4e715e0789ea-tls-certs\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.893543 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.893521    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbl4z\" (UniqueName: \"kubernetes.io/projected/4bb0de17-a082-4c75-a783-4e715e0789ea-kube-api-access-sbl4z\") pod \"rate-test-model-d50ef4-kserve-7747b46d75-gtl95\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") " pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:07.987348 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:07.987292    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:08.112055 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:08.112030    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"]
Jul 25 01:29:08.113615 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:29:08.113581    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bb0de17_a082_4c75_a783_4e715e0789ea.slice/crio-2adbd02149e6887d2a42aa4f36c500c066767c77f7048fdb814cdec96c384840 WatchSource:0}: Error finding container 2adbd02149e6887d2a42aa4f36c500c066767c77f7048fdb814cdec96c384840: Status 404 returned error can't find the container with id 2adbd02149e6887d2a42aa4f36c500c066767c77f7048fdb814cdec96c384840
Jul 25 01:29:08.200764 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:08.200738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95" event={"ID":"4bb0de17-a082-4c75-a783-4e715e0789ea","Type":"ContainerStarted","Data":"efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b"}
Jul 25 01:29:08.200893 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:08.200772    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95" event={"ID":"4bb0de17-a082-4c75-a783-4e715e0789ea","Type":"ContainerStarted","Data":"2adbd02149e6887d2a42aa4f36c500c066767c77f7048fdb814cdec96c384840"}
Jul 25 01:29:08.217146 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:08.217102    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95" podStartSLOduration=1.217083281 podStartE2EDuration="1.217083281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:29:08.216086198 +0000 UTC m=+3170.604608604" watchObservedRunningTime="2026-07-25 01:29:08.217083281 +0000 UTC m=+3170.605605685"
Jul 25 01:29:11.886059 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:11.886021    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:29:17.987489 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:17.987443    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:17.987911 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:17.987511    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:18.000015 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:17.999980    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:18.256508 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:18.256430    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:29:43.793680 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:43.793631    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:29:58.487445 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:29:58.487406    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:30:15.792812 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:15.792764    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:30:21.180856 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:21.180815    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"]
Jul 25 01:30:30.999869 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:30.999818    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:30:36.199788 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.199726    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95" podUID="4bb0de17-a082-4c75-a783-4e715e0789ea" containerName="main" containerID="cri-o://efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b" gracePeriod=45
Jul 25 01:30:36.439304 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.439282    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:30:36.547225 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.547169    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bb0de17-a082-4c75-a783-4e715e0789ea-tls-certs\") pod \"4bb0de17-a082-4c75-a783-4e715e0789ea\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") "
Jul 25 01:30:36.547225 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.547205    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-dshm\") pod \"4bb0de17-a082-4c75-a783-4e715e0789ea\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") "
Jul 25 01:30:36.547225 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.547222    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-home\") pod \"4bb0de17-a082-4c75-a783-4e715e0789ea\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") "
Jul 25 01:30:36.547423 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.547243    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-model-cache\") pod \"4bb0de17-a082-4c75-a783-4e715e0789ea\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") "
Jul 25 01:30:36.547423 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.547309    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbl4z\" (UniqueName: \"kubernetes.io/projected/4bb0de17-a082-4c75-a783-4e715e0789ea-kube-api-access-sbl4z\") pod \"4bb0de17-a082-4c75-a783-4e715e0789ea\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") "
Jul 25 01:30:36.547423 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.547343    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-tmp-dir\") pod \"4bb0de17-a082-4c75-a783-4e715e0789ea\" (UID: \"4bb0de17-a082-4c75-a783-4e715e0789ea\") "
Jul 25 01:30:36.547584 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.547466    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-home" (OuterVolumeSpecName: "home") pod "4bb0de17-a082-4c75-a783-4e715e0789ea" (UID: "4bb0de17-a082-4c75-a783-4e715e0789ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:30:36.547584 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.547547    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-model-cache" (OuterVolumeSpecName: "model-cache") pod "4bb0de17-a082-4c75-a783-4e715e0789ea" (UID: "4bb0de17-a082-4c75-a783-4e715e0789ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:30:36.547688 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.547640    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4bb0de17-a082-4c75-a783-4e715e0789ea" (UID: "4bb0de17-a082-4c75-a783-4e715e0789ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:30:36.549241 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.549223    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-dshm" (OuterVolumeSpecName: "dshm") pod "4bb0de17-a082-4c75-a783-4e715e0789ea" (UID: "4bb0de17-a082-4c75-a783-4e715e0789ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:30:36.549334 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.549316    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bb0de17-a082-4c75-a783-4e715e0789ea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4bb0de17-a082-4c75-a783-4e715e0789ea" (UID: "4bb0de17-a082-4c75-a783-4e715e0789ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:30:36.549380 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.549360    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bb0de17-a082-4c75-a783-4e715e0789ea-kube-api-access-sbl4z" (OuterVolumeSpecName: "kube-api-access-sbl4z") pod "4bb0de17-a082-4c75-a783-4e715e0789ea" (UID: "4bb0de17-a082-4c75-a783-4e715e0789ea"). InnerVolumeSpecName "kube-api-access-sbl4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:30:36.589462 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.589438    2566 generic.go:356] "Generic (PLEG): container finished" podID="4bb0de17-a082-4c75-a783-4e715e0789ea" containerID="efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b" exitCode=0
Jul 25 01:30:36.589565 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.589476    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"
Jul 25 01:30:36.589565 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.589531    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95" event={"ID":"4bb0de17-a082-4c75-a783-4e715e0789ea","Type":"ContainerDied","Data":"efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b"}
Jul 25 01:30:36.589691 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.589576    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95" event={"ID":"4bb0de17-a082-4c75-a783-4e715e0789ea","Type":"ContainerDied","Data":"2adbd02149e6887d2a42aa4f36c500c066767c77f7048fdb814cdec96c384840"}
Jul 25 01:30:36.589691 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.589608    2566 scope.go:117] "RemoveContainer" containerID="efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b"
Jul 25 01:30:36.597239 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.597223    2566 scope.go:117] "RemoveContainer" containerID="efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b"
Jul 25 01:30:36.597478 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:30:36.597461    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b\": container with ID starting with efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b not found: ID does not exist" containerID="efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b"
Jul 25 01:30:36.597522 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.597485    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b"} err="failed to get container status \"efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b\": rpc error: code = NotFound desc = could not find container \"efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b\": container with ID starting with efea6764e9e2a8eba8388ee0e788173559b1f4fe4031e4636ae4bfd91aafb70b not found: ID does not exist"
Jul 25 01:30:36.608441 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.608421    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"]
Jul 25 01:30:36.611482 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.611463    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d50ef4/rate-test-model-d50ef4-kserve-7747b46d75-gtl95"]
Jul 25 01:30:36.648837 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.648818    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:30:36.648889 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.648837    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbl4z\" (UniqueName: \"kubernetes.io/projected/4bb0de17-a082-4c75-a783-4e715e0789ea-kube-api-access-sbl4z\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:30:36.648889 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.648863    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:30:36.648889 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.648874    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bb0de17-a082-4c75-a783-4e715e0789ea-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:30:36.648889 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.648881    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:30:36.648889 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:36.648888    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bb0de17-a082-4c75-a783-4e715e0789ea-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:30:38.174216 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:38.174178    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bb0de17-a082-4c75-a783-4e715e0789ea" path="/var/lib/kubelet/pods/4bb0de17-a082-4c75-a783-4e715e0789ea/volumes"
Jul 25 01:30:44.586112 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.586081    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"]
Jul 25 01:30:44.586466 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.586385    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb0de17-a082-4c75-a783-4e715e0789ea" containerName="main"
Jul 25 01:30:44.586466 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.586396    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb0de17-a082-4c75-a783-4e715e0789ea" containerName="main"
Jul 25 01:30:44.586466 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.586444    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bb0de17-a082-4c75-a783-4e715e0789ea" containerName="main"
Jul 25 01:30:44.590689 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.590666    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:30:44.592884 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.592865    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-7b2fa1\""
Jul 25 01:30:44.599462 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.599437    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"]
Jul 25 01:30:44.602451 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.602427    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"]
Jul 25 01:30:44.605784 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.605763    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:30:44.608001 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.607978    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-7b2fa1-dockercfg-ldb4c\""
Jul 25 01:30:44.616149 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.616131    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"]
Jul 25 01:30:44.705299 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.705269    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnzhk\" (UniqueName: \"kubernetes.io/projected/12ca0984-42f8-4e80-97ad-39707111c040-kube-api-access-jnzhk\") pod \"payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45\" (UID: \"12ca0984-42f8-4e80-97ad-39707111c040\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:30:44.705419 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.705327    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ca0984-42f8-4e80-97ad-39707111c040-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45\" (UID: \"12ca0984-42f8-4e80-97ad-39707111c040\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:30:44.705419 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.705376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eccd602a-24d2-4069-b102-65e90f7424ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2\" (UID: \"eccd602a-24d2-4069-b102-65e90f7424ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:30:44.705517 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.705477    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rbjs\" (UniqueName: \"kubernetes.io/projected/eccd602a-24d2-4069-b102-65e90f7424ea-kube-api-access-7rbjs\") pod \"payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2\" (UID: \"eccd602a-24d2-4069-b102-65e90f7424ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:30:44.806612 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.806586    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnzhk\" (UniqueName: \"kubernetes.io/projected/12ca0984-42f8-4e80-97ad-39707111c040-kube-api-access-jnzhk\") pod \"payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45\" (UID: \"12ca0984-42f8-4e80-97ad-39707111c040\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:30:44.806754 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.806625    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ca0984-42f8-4e80-97ad-39707111c040-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45\" (UID: \"12ca0984-42f8-4e80-97ad-39707111c040\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:30:44.806754 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.806644    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eccd602a-24d2-4069-b102-65e90f7424ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2\" (UID: \"eccd602a-24d2-4069-b102-65e90f7424ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:30:44.806754 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.806677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rbjs\" (UniqueName: \"kubernetes.io/projected/eccd602a-24d2-4069-b102-65e90f7424ea-kube-api-access-7rbjs\") pod \"payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2\" (UID: \"eccd602a-24d2-4069-b102-65e90f7424ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:30:44.807391 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.807369    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eccd602a-24d2-4069-b102-65e90f7424ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2\" (UID: \"eccd602a-24d2-4069-b102-65e90f7424ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:30:44.807477 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.807375    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ca0984-42f8-4e80-97ad-39707111c040-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45\" (UID: \"12ca0984-42f8-4e80-97ad-39707111c040\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:30:44.815043 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.815019    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rbjs\" (UniqueName: \"kubernetes.io/projected/eccd602a-24d2-4069-b102-65e90f7424ea-kube-api-access-7rbjs\") pod \"payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2\" (UID: \"eccd602a-24d2-4069-b102-65e90f7424ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:30:44.815154 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.815022    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnzhk\" (UniqueName: \"kubernetes.io/projected/12ca0984-42f8-4e80-97ad-39707111c040-kube-api-access-jnzhk\") pod \"payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45\" (UID: \"12ca0984-42f8-4e80-97ad-39707111c040\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:30:44.901798 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.901771    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:30:44.916896 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:44.916875    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:30:45.062759 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:45.062734    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"]
Jul 25 01:30:45.065313 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:30:45.065288    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccd602a_24d2_4069_b102_65e90f7424ea.slice/crio-8955f0323f62f8384c86ba02e170665de233ba4a3d752f04a23d3f33ff720b16 WatchSource:0}: Error finding container 8955f0323f62f8384c86ba02e170665de233ba4a3d752f04a23d3f33ff720b16: Status 404 returned error can't find the container with id 8955f0323f62f8384c86ba02e170665de233ba4a3d752f04a23d3f33ff720b16
Jul 25 01:30:45.244395 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:45.244333    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"]
Jul 25 01:30:45.247474 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:30:45.247449    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ca0984_42f8_4e80_97ad_39707111c040.slice/crio-0822fbdfcc8549f66cf95aa40c60738b11e9740abac95075a016a32e6426b36a WatchSource:0}: Error finding container 0822fbdfcc8549f66cf95aa40c60738b11e9740abac95075a016a32e6426b36a: Status 404 returned error can't find the container with id 0822fbdfcc8549f66cf95aa40c60738b11e9740abac95075a016a32e6426b36a
Jul 25 01:30:45.631109 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:45.631055    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2" event={"ID":"eccd602a-24d2-4069-b102-65e90f7424ea","Type":"ContainerStarted","Data":"8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86"}
Jul 25 01:30:45.631544 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:45.631201    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:30:45.631544 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:45.631236    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2" event={"ID":"eccd602a-24d2-4069-b102-65e90f7424ea","Type":"ContainerStarted","Data":"8955f0323f62f8384c86ba02e170665de233ba4a3d752f04a23d3f33ff720b16"}
Jul 25 01:30:45.632654 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:45.632627    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45" event={"ID":"12ca0984-42f8-4e80-97ad-39707111c040","Type":"ContainerStarted","Data":"a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199"}
Jul 25 01:30:45.632781 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:45.632674    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45" event={"ID":"12ca0984-42f8-4e80-97ad-39707111c040","Type":"ContainerStarted","Data":"0822fbdfcc8549f66cf95aa40c60738b11e9740abac95075a016a32e6426b36a"}
Jul 25 01:30:45.632781 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:45.632702    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:30:45.647324 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:45.647259    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2" podStartSLOduration=1.647241773 podStartE2EDuration="1.647241773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:30:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:30:45.645348083 +0000 UTC m=+3268.033870487" watchObservedRunningTime="2026-07-25 01:30:45.647241773 +0000 UTC m=+3268.035764176"
Jul 25 01:30:45.667729 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:45.667689    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45" podStartSLOduration=1.667678557 podStartE2EDuration="1.667678557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:30:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:30:45.665807459 +0000 UTC m=+3268.054329864" watchObservedRunningTime="2026-07-25 01:30:45.667678557 +0000 UTC m=+3268.056200959"
Jul 25 01:30:52.283493 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:52.283454    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:30:56.638944 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:56.638908    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:30:56.639378 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:30:56.638992    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:31:03.590791 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.590749    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"]
Jul 25 01:31:03.594183 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.594165    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:31:03.596497 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.596475    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-6bf6cd-dockercfg-dt5qr\""
Jul 25 01:31:03.596863 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.596849    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-6bf6cd\""
Jul 25 01:31:03.606689 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.606667    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"]
Jul 25 01:31:03.617304 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.617283    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"]
Jul 25 01:31:03.620425 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.620409    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:31:03.630013 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.629994    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"]
Jul 25 01:31:03.748315 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.748283    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/171f0eb2-9bbe-4b19-afc4-a223b6789316-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp\" (UID: \"171f0eb2-9bbe-4b19-afc4-a223b6789316\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:31:03.748315 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.748318    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ph7h\" (UniqueName: \"kubernetes.io/projected/171f0eb2-9bbe-4b19-afc4-a223b6789316-kube-api-access-6ph7h\") pod \"payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp\" (UID: \"171f0eb2-9bbe-4b19-afc4-a223b6789316\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:31:03.748516 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.748341    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg\" (UID: \"9a389fb8-76d3-4453-9a13-01dbf5ef4e28\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:31:03.748516 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.748440    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkb5l\" (UniqueName: \"kubernetes.io/projected/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-kube-api-access-tkb5l\") pod \"payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg\" (UID: \"9a389fb8-76d3-4453-9a13-01dbf5ef4e28\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:31:03.849205 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.849138    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg\" (UID: \"9a389fb8-76d3-4453-9a13-01dbf5ef4e28\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:31:03.849205 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.849174    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkb5l\" (UniqueName: \"kubernetes.io/projected/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-kube-api-access-tkb5l\") pod \"payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg\" (UID: \"9a389fb8-76d3-4453-9a13-01dbf5ef4e28\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:31:03.849363 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.849228    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/171f0eb2-9bbe-4b19-afc4-a223b6789316-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp\" (UID: \"171f0eb2-9bbe-4b19-afc4-a223b6789316\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:31:03.849363 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.849246    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ph7h\" (UniqueName: \"kubernetes.io/projected/171f0eb2-9bbe-4b19-afc4-a223b6789316-kube-api-access-6ph7h\") pod \"payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp\" (UID: \"171f0eb2-9bbe-4b19-afc4-a223b6789316\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:31:03.849892 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.849875    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg\" (UID: \"9a389fb8-76d3-4453-9a13-01dbf5ef4e28\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:31:03.849937 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.849886    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/171f0eb2-9bbe-4b19-afc4-a223b6789316-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp\" (UID: \"171f0eb2-9bbe-4b19-afc4-a223b6789316\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:31:03.857980 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.857959    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ph7h\" (UniqueName: \"kubernetes.io/projected/171f0eb2-9bbe-4b19-afc4-a223b6789316-kube-api-access-6ph7h\") pod \"payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp\" (UID: \"171f0eb2-9bbe-4b19-afc4-a223b6789316\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:31:03.858094 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.858001    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkb5l\" (UniqueName: \"kubernetes.io/projected/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-kube-api-access-tkb5l\") pod \"payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg\" (UID: \"9a389fb8-76d3-4453-9a13-01dbf5ef4e28\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:31:03.904387 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.904366    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:31:03.930295 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:03.930268    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:31:04.039515 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:04.039489    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"]
Jul 25 01:31:04.041993 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:31:04.041961    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod171f0eb2_9bbe_4b19_afc4_a223b6789316.slice/crio-c4ebf2ebf2b0291bddacf85535819f7004f131738c46df6b96dd44c2789a5ba1 WatchSource:0}: Error finding container c4ebf2ebf2b0291bddacf85535819f7004f131738c46df6b96dd44c2789a5ba1: Status 404 returned error can't find the container with id c4ebf2ebf2b0291bddacf85535819f7004f131738c46df6b96dd44c2789a5ba1
Jul 25 01:31:04.067960 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:04.067935    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"]
Jul 25 01:31:04.078055 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:31:04.078029    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a389fb8_76d3_4453_9a13_01dbf5ef4e28.slice/crio-8d612aa8dd5bf3c7fadceb9efd2a6ce0ec70500467e9befc9f67b82c2a047721 WatchSource:0}: Error finding container 8d612aa8dd5bf3c7fadceb9efd2a6ce0ec70500467e9befc9f67b82c2a047721: Status 404 returned error can't find the container with id 8d612aa8dd5bf3c7fadceb9efd2a6ce0ec70500467e9befc9f67b82c2a047721
Jul 25 01:31:04.725374 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:04.725338    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg" event={"ID":"9a389fb8-76d3-4453-9a13-01dbf5ef4e28","Type":"ContainerStarted","Data":"cc54bbe153a5448e8b65491fdc9609ac7d4301c785b8c0d98ae27c6ad7acc9c1"}
Jul 25 01:31:04.725808 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:04.725402    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg" event={"ID":"9a389fb8-76d3-4453-9a13-01dbf5ef4e28","Type":"ContainerStarted","Data":"8d612aa8dd5bf3c7fadceb9efd2a6ce0ec70500467e9befc9f67b82c2a047721"}
Jul 25 01:31:04.725808 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:04.725531    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:31:04.726821 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:04.726797    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp" event={"ID":"171f0eb2-9bbe-4b19-afc4-a223b6789316","Type":"ContainerStarted","Data":"8cbf3b316f4c1ea814bb6934fc78eee8364cdaa30588296cbc781866e2cc6567"}
Jul 25 01:31:04.726934 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:04.726845    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp" event={"ID":"171f0eb2-9bbe-4b19-afc4-a223b6789316","Type":"ContainerStarted","Data":"c4ebf2ebf2b0291bddacf85535819f7004f131738c46df6b96dd44c2789a5ba1"}
Jul 25 01:31:04.726934 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:04.726874    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:31:04.741676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:04.741634    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg" podStartSLOduration=1.741620921 podStartE2EDuration="1.741620921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:31:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:31:04.740412422 +0000 UTC m=+3287.128934827" watchObservedRunningTime="2026-07-25 01:31:04.741620921 +0000 UTC m=+3287.130143302"
Jul 25 01:31:04.755096 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:04.755035    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp" podStartSLOduration=1.755020885 podStartE2EDuration="1.755020885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:31:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:31:04.754591523 +0000 UTC m=+3287.143113927" watchObservedRunningTime="2026-07-25 01:31:04.755020885 +0000 UTC m=+3287.143543287"
Jul 25 01:31:10.088904 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:10.088863    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:31:15.733255 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:15.733212    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:31:15.733709 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:15.733408    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:31:24.679090 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:24.679039    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg_9a389fb8-76d3-4453-9a13-01dbf5ef4e28/payload-processing/0.log"
Jul 25 01:31:30.291420 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:30.291382    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:31:43.171600 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.171513    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r8qvr"]
Jul 25 01:31:43.175364 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.175339    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:43.177600 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.177570    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9jd7t\""
Jul 25 01:31:43.185907 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.185877    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r8qvr"]
Jul 25 01:31:43.242638 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.242612    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw5cw\" (UniqueName: \"kubernetes.io/projected/912182df-ee3d-41ed-8b06-f73f8d752d62-kube-api-access-tw5cw\") pod \"kuadrant-operator-catalog-r8qvr\" (UID: \"912182df-ee3d-41ed-8b06-f73f8d752d62\") " pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:43.343860 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.343832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5cw\" (UniqueName: \"kubernetes.io/projected/912182df-ee3d-41ed-8b06-f73f8d752d62-kube-api-access-tw5cw\") pod \"kuadrant-operator-catalog-r8qvr\" (UID: \"912182df-ee3d-41ed-8b06-f73f8d752d62\") " pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:43.352205 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.352175    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw5cw\" (UniqueName: \"kubernetes.io/projected/912182df-ee3d-41ed-8b06-f73f8d752d62-kube-api-access-tw5cw\") pod \"kuadrant-operator-catalog-r8qvr\" (UID: \"912182df-ee3d-41ed-8b06-f73f8d752d62\") " pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:43.487297 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.487236    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:43.611246 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.611122    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r8qvr"]
Jul 25 01:31:43.613975 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:31:43.613949    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod912182df_ee3d_41ed_8b06_f73f8d752d62.slice/crio-338f771128c40753eb00612cc9c4515a4256104a624183ef0cb618e0e9391b36 WatchSource:0}: Error finding container 338f771128c40753eb00612cc9c4515a4256104a624183ef0cb618e0e9391b36: Status 404 returned error can't find the container with id 338f771128c40753eb00612cc9c4515a4256104a624183ef0cb618e0e9391b36
Jul 25 01:31:43.615261 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.615243    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:31:43.912200 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:43.912166    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr" event={"ID":"912182df-ee3d-41ed-8b06-f73f8d752d62","Type":"ContainerStarted","Data":"338f771128c40753eb00612cc9c4515a4256104a624183ef0cb618e0e9391b36"}
Jul 25 01:31:45.924560 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:45.924521    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr" event={"ID":"912182df-ee3d-41ed-8b06-f73f8d752d62","Type":"ContainerStarted","Data":"a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14"}
Jul 25 01:31:45.940230 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:45.940180    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr" podStartSLOduration=0.990635994 podStartE2EDuration="2.940164364s" totalImagesPullingTime="1.94952837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:31:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:31:45.938860774 +0000 UTC m=+3328.327383179" watchObservedRunningTime="2026-07-25 01:31:45.940164364 +0000 UTC m=+3328.328686816"
Jul 25 01:31:53.487662 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:53.487623    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:53.488148 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:53.487714    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:53.510557 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:53.510523    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:53.982580 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:53.982554    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:54.018378 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:54.018352    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r8qvr"]
Jul 25 01:31:55.970513 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:55.970452    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr" podUID="912182df-ee3d-41ed-8b06-f73f8d752d62" containerName="registry-server" containerID="cri-o://a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14" gracePeriod=2
Jul 25 01:31:56.215244 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.215220    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:56.281873 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.281810    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:31:56.345526 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.345503    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw5cw\" (UniqueName: \"kubernetes.io/projected/912182df-ee3d-41ed-8b06-f73f8d752d62-kube-api-access-tw5cw\") pod \"912182df-ee3d-41ed-8b06-f73f8d752d62\" (UID: \"912182df-ee3d-41ed-8b06-f73f8d752d62\") "
Jul 25 01:31:56.347391 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.347368    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/912182df-ee3d-41ed-8b06-f73f8d752d62-kube-api-access-tw5cw" (OuterVolumeSpecName: "kube-api-access-tw5cw") pod "912182df-ee3d-41ed-8b06-f73f8d752d62" (UID: "912182df-ee3d-41ed-8b06-f73f8d752d62"). InnerVolumeSpecName "kube-api-access-tw5cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:31:56.446223 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.446200    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw5cw\" (UniqueName: \"kubernetes.io/projected/912182df-ee3d-41ed-8b06-f73f8d752d62-kube-api-access-tw5cw\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:31:56.976940 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.976905    2566 generic.go:356] "Generic (PLEG): container finished" podID="912182df-ee3d-41ed-8b06-f73f8d752d62" containerID="a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14" exitCode=0
Jul 25 01:31:56.977469 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.976954    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr"
Jul 25 01:31:56.977469 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.977013    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr" event={"ID":"912182df-ee3d-41ed-8b06-f73f8d752d62","Type":"ContainerDied","Data":"a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14"}
Jul 25 01:31:56.977469 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.977107    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r8qvr" event={"ID":"912182df-ee3d-41ed-8b06-f73f8d752d62","Type":"ContainerDied","Data":"338f771128c40753eb00612cc9c4515a4256104a624183ef0cb618e0e9391b36"}
Jul 25 01:31:56.977469 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.977143    2566 scope.go:117] "RemoveContainer" containerID="a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14"
Jul 25 01:31:56.987680 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.987659    2566 scope.go:117] "RemoveContainer" containerID="a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14"
Jul 25 01:31:56.987958 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:31:56.987934    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14\": container with ID starting with a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14 not found: ID does not exist" containerID="a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14"
Jul 25 01:31:56.988032 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.987974    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14"} err="failed to get container status \"a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14\": rpc error: code = NotFound desc = could not find container \"a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14\": container with ID starting with a133e2fadffe82a3f013f66aee7ddd21add52bf382ed4e4ecd3b722240ce9e14 not found: ID does not exist"
Jul 25 01:31:56.997617 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:56.997590    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r8qvr"]
Jul 25 01:31:57.001244 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:57.001218    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r8qvr"]
Jul 25 01:31:58.171702 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:31:58.171669    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="912182df-ee3d-41ed-8b06-f73f8d752d62" path="/var/lib/kubelet/pods/912182df-ee3d-41ed-8b06-f73f8d752d62/volumes"
Jul 25 01:32:06.625220 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:06.625183    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45_12ca0984-42f8-4e80-97ad-39707111c040/payload-processing/0.log"
Jul 25 01:32:06.945440 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:06.945358    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg_9a389fb8-76d3-4453-9a13-01dbf5ef4e28/payload-processing/0.log"
Jul 25 01:32:16.975441 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:16.975409    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"]
Jul 25 01:32:16.975785 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:16.975750    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="912182df-ee3d-41ed-8b06-f73f8d752d62" containerName="registry-server"
Jul 25 01:32:16.975785 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:16.975761    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="912182df-ee3d-41ed-8b06-f73f8d752d62" containerName="registry-server"
Jul 25 01:32:16.975860 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:16.975820    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="912182df-ee3d-41ed-8b06-f73f8d752d62" containerName="registry-server"
Jul 25 01:32:16.978771 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:16.978754    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:16.981174 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:16.981152    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-03dd34\""
Jul 25 01:32:16.991172 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:16.991148    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"]
Jul 25 01:32:17.028337 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.028312    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"]
Jul 25 01:32:17.031465 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.031451    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:17.033789 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.033771    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-03dd34-dockercfg-kgbrq\""
Jul 25 01:32:17.041091 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.041053    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"]
Jul 25 01:32:17.101092 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.101046    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb659ab1-f62b-4b3d-be84-42123b161cfb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr\" (UID: \"fb659ab1-f62b-4b3d-be84-42123b161cfb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:17.101210 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.101098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bwct\" (UniqueName: \"kubernetes.io/projected/fb659ab1-f62b-4b3d-be84-42123b161cfb-kube-api-access-5bwct\") pod \"payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr\" (UID: \"fb659ab1-f62b-4b3d-be84-42123b161cfb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:17.202510 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.202480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m\" (UID: \"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:17.202640 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.202528    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7vt6\" (UniqueName: \"kubernetes.io/projected/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-kube-api-access-z7vt6\") pod \"payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m\" (UID: \"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:17.202640 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.202582    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb659ab1-f62b-4b3d-be84-42123b161cfb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr\" (UID: \"fb659ab1-f62b-4b3d-be84-42123b161cfb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:17.202640 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.202601    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bwct\" (UniqueName: \"kubernetes.io/projected/fb659ab1-f62b-4b3d-be84-42123b161cfb-kube-api-access-5bwct\") pod \"payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr\" (UID: \"fb659ab1-f62b-4b3d-be84-42123b161cfb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:17.203340 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.203321    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb659ab1-f62b-4b3d-be84-42123b161cfb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr\" (UID: \"fb659ab1-f62b-4b3d-be84-42123b161cfb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:17.210515 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.210497    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bwct\" (UniqueName: \"kubernetes.io/projected/fb659ab1-f62b-4b3d-be84-42123b161cfb-kube-api-access-5bwct\") pod \"payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr\" (UID: \"fb659ab1-f62b-4b3d-be84-42123b161cfb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:17.289146 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.289093    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:17.303922 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.303897    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m\" (UID: \"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:17.304032 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.303950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7vt6\" (UniqueName: \"kubernetes.io/projected/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-kube-api-access-z7vt6\") pod \"payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m\" (UID: \"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:17.304759 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.304735    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m\" (UID: \"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:17.315704 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.315681    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7vt6\" (UniqueName: \"kubernetes.io/projected/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-kube-api-access-z7vt6\") pod \"payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m\" (UID: \"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:17.342317 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.342282    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:17.425738 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.425711    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"]
Jul 25 01:32:17.428226 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:32:17.428203    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb659ab1_f62b_4b3d_be84_42123b161cfb.slice/crio-5391c9c092382cf14daab6ec227893f35ee52dfd5b33284a89fc4e9c9e87ddf4 WatchSource:0}: Error finding container 5391c9c092382cf14daab6ec227893f35ee52dfd5b33284a89fc4e9c9e87ddf4: Status 404 returned error can't find the container with id 5391c9c092382cf14daab6ec227893f35ee52dfd5b33284a89fc4e9c9e87ddf4
Jul 25 01:32:17.486845 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:17.486767    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"]
Jul 25 01:32:17.489521 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:32:17.489494    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bab6a9d_0a41_4b4a_aa64_deeb6a7acc8e.slice/crio-284438b835d4ecfc082e586e1d14a231df36f99dcddf925f644b4dcfcc6d169a WatchSource:0}: Error finding container 284438b835d4ecfc082e586e1d14a231df36f99dcddf925f644b4dcfcc6d169a: Status 404 returned error can't find the container with id 284438b835d4ecfc082e586e1d14a231df36f99dcddf925f644b4dcfcc6d169a
Jul 25 01:32:18.082128 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:18.082095    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m" event={"ID":"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e","Type":"ContainerStarted","Data":"2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea"}
Jul 25 01:32:18.082529 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:18.082152    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m" event={"ID":"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e","Type":"ContainerStarted","Data":"284438b835d4ecfc082e586e1d14a231df36f99dcddf925f644b4dcfcc6d169a"}
Jul 25 01:32:18.082529 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:18.082207    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:18.083564 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:18.083546    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr" event={"ID":"fb659ab1-f62b-4b3d-be84-42123b161cfb","Type":"ContainerStarted","Data":"cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa"}
Jul 25 01:32:18.083616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:18.083579    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr" event={"ID":"fb659ab1-f62b-4b3d-be84-42123b161cfb","Type":"ContainerStarted","Data":"5391c9c092382cf14daab6ec227893f35ee52dfd5b33284a89fc4e9c9e87ddf4"}
Jul 25 01:32:18.083616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:18.083598    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:18.101422 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:18.101385    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m" podStartSLOduration=2.101373506 podStartE2EDuration="2.101373506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:32:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:32:18.099764195 +0000 UTC m=+3360.488286597" watchObservedRunningTime="2026-07-25 01:32:18.101373506 +0000 UTC m=+3360.489895908"
Jul 25 01:32:18.114002 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:18.113962    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr" podStartSLOduration=2.11395045 podStartE2EDuration="2.11395045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:32:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:32:18.113122039 +0000 UTC m=+3360.501644443" watchObservedRunningTime="2026-07-25 01:32:18.11395045 +0000 UTC m=+3360.502472853"
Jul 25 01:32:21.889854 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:21.889820    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:32:29.089762 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:29.089719    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:29.090368 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:29.090039    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:42.233693 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.233659    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"]
Jul 25 01:32:42.234185 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.233904    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m" podUID="0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e" containerName="payload-processing" containerID="cri-o://2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea" gracePeriod=30
Jul 25 01:32:42.239951 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.239921    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"]
Jul 25 01:32:42.240129 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.240105    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr" podUID="fb659ab1-f62b-4b3d-be84-42123b161cfb" containerName="payload-pre-processing" containerID="cri-o://cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa" gracePeriod=30
Jul 25 01:32:42.517157 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.517134    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:42.519607 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.519590    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:42.596492 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.596471    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb659ab1-f62b-4b3d-be84-42123b161cfb-plugins-config-volume\") pod \"fb659ab1-f62b-4b3d-be84-42123b161cfb\" (UID: \"fb659ab1-f62b-4b3d-be84-42123b161cfb\") "
Jul 25 01:32:42.596616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.596513    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7vt6\" (UniqueName: \"kubernetes.io/projected/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-kube-api-access-z7vt6\") pod \"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e\" (UID: \"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e\") "
Jul 25 01:32:42.596616 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.596536    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bwct\" (UniqueName: \"kubernetes.io/projected/fb659ab1-f62b-4b3d-be84-42123b161cfb-kube-api-access-5bwct\") pod \"fb659ab1-f62b-4b3d-be84-42123b161cfb\" (UID: \"fb659ab1-f62b-4b3d-be84-42123b161cfb\") "
Jul 25 01:32:42.596737 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.596711    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-plugins-config-volume\") pod \"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e\" (UID: \"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e\") "
Jul 25 01:32:42.596881 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.596861    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb659ab1-f62b-4b3d-be84-42123b161cfb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fb659ab1-f62b-4b3d-be84-42123b161cfb" (UID: "fb659ab1-f62b-4b3d-be84-42123b161cfb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:32:42.596982 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.596965    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb659ab1-f62b-4b3d-be84-42123b161cfb-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:32:42.597049 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.597025    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e" (UID: "0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:32:42.598471 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.598453    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb659ab1-f62b-4b3d-be84-42123b161cfb-kube-api-access-5bwct" (OuterVolumeSpecName: "kube-api-access-5bwct") pod "fb659ab1-f62b-4b3d-be84-42123b161cfb" (UID: "fb659ab1-f62b-4b3d-be84-42123b161cfb"). InnerVolumeSpecName "kube-api-access-5bwct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:32:42.598531 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.598477    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-kube-api-access-z7vt6" (OuterVolumeSpecName: "kube-api-access-z7vt6") pod "0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e" (UID: "0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e"). InnerVolumeSpecName "kube-api-access-z7vt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:32:42.697910 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.697878    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:32:42.697910 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.697908    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7vt6\" (UniqueName: \"kubernetes.io/projected/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e-kube-api-access-z7vt6\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:32:42.697910 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:42.697918    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bwct\" (UniqueName: \"kubernetes.io/projected/fb659ab1-f62b-4b3d-be84-42123b161cfb-kube-api-access-5bwct\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:32:43.203923 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.203892    2566 generic.go:356] "Generic (PLEG): container finished" podID="0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e" containerID="2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea" exitCode=0
Jul 25 01:32:43.204055 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.203935    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"
Jul 25 01:32:43.204055 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.203984    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m" event={"ID":"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e","Type":"ContainerDied","Data":"2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea"}
Jul 25 01:32:43.204055 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.204041    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m" event={"ID":"0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e","Type":"ContainerDied","Data":"284438b835d4ecfc082e586e1d14a231df36f99dcddf925f644b4dcfcc6d169a"}
Jul 25 01:32:43.204234 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.204103    2566 scope.go:117] "RemoveContainer" containerID="2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea"
Jul 25 01:32:43.205641 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.205611    2566 generic.go:356] "Generic (PLEG): container finished" podID="fb659ab1-f62b-4b3d-be84-42123b161cfb" containerID="cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa" exitCode=0
Jul 25 01:32:43.205744 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.205659    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"
Jul 25 01:32:43.205744 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.205672    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr" event={"ID":"fb659ab1-f62b-4b3d-be84-42123b161cfb","Type":"ContainerDied","Data":"cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa"}
Jul 25 01:32:43.205744 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.205719    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr" event={"ID":"fb659ab1-f62b-4b3d-be84-42123b161cfb","Type":"ContainerDied","Data":"5391c9c092382cf14daab6ec227893f35ee52dfd5b33284a89fc4e9c9e87ddf4"}
Jul 25 01:32:43.214559 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.214530    2566 scope.go:117] "RemoveContainer" containerID="2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea"
Jul 25 01:32:43.214838 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:32:43.214814    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea\": container with ID starting with 2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea not found: ID does not exist" containerID="2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea"
Jul 25 01:32:43.214934 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.214847    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea"} err="failed to get container status \"2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea\": rpc error: code = NotFound desc = could not find container \"2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea\": container with ID starting with 2e14e8abe512ef599070f29e7d1854627464bac5d638936f9deb8784805f12ea not found: ID does not exist"
Jul 25 01:32:43.214934 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.214871    2566 scope.go:117] "RemoveContainer" containerID="cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa"
Jul 25 01:32:43.222929 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.222909    2566 scope.go:117] "RemoveContainer" containerID="cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa"
Jul 25 01:32:43.223273 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:32:43.223246    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa\": container with ID starting with cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa not found: ID does not exist" containerID="cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa"
Jul 25 01:32:43.223367 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.223279    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa"} err="failed to get container status \"cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa\": rpc error: code = NotFound desc = could not find container \"cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa\": container with ID starting with cb08e4298966548d6b51c485c3daf89ad644d181e40c7b759c406354cbecc9fa not found: ID does not exist"
Jul 25 01:32:43.226641 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.226613    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"]
Jul 25 01:32:43.228124 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.228092    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-03dd34-6dd4d86cd4-rsf8m"]
Jul 25 01:32:43.236342 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.236320    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"]
Jul 25 01:32:43.240508 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:43.240486    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-03dd34-54dc844bdd-b5llr"]
Jul 25 01:32:44.172198 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:44.172170    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e" path="/var/lib/kubelet/pods/0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e/volumes"
Jul 25 01:32:44.172547 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:44.172531    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb659ab1-f62b-4b3d-be84-42123b161cfb" path="/var/lib/kubelet/pods/fb659ab1-f62b-4b3d-be84-42123b161cfb/volumes"
Jul 25 01:32:47.186548 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:47.186507    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:32:57.206057 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.206010    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"]
Jul 25 01:32:57.206580 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.206556    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e" containerName="payload-processing"
Jul 25 01:32:57.206662 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.206583    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e" containerName="payload-processing"
Jul 25 01:32:57.206662 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.206603    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb659ab1-f62b-4b3d-be84-42123b161cfb" containerName="payload-pre-processing"
Jul 25 01:32:57.206662 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.206612    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb659ab1-f62b-4b3d-be84-42123b161cfb" containerName="payload-pre-processing"
Jul 25 01:32:57.206813 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.206713    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bab6a9d-0a41-4b4a-aa64-deeb6a7acc8e" containerName="payload-processing"
Jul 25 01:32:57.206813 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.206728    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb659ab1-f62b-4b3d-be84-42123b161cfb" containerName="payload-pre-processing"
Jul 25 01:32:57.211391 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.211363    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"
Jul 25 01:32:57.214046 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.214015    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:32:57.214179 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.214044    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:32:57.215815 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.215784    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"]
Jul 25 01:32:57.305404 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.305377    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d43c4b-0998-42c8-80ba-1c087e86e146-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z\" (UID: \"36d43c4b-0998-42c8-80ba-1c087e86e146\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"
Jul 25 01:32:57.406321 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.406292    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d43c4b-0998-42c8-80ba-1c087e86e146-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z\" (UID: \"36d43c4b-0998-42c8-80ba-1c087e86e146\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"
Jul 25 01:32:57.406938 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.406918    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d43c4b-0998-42c8-80ba-1c087e86e146-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z\" (UID: \"36d43c4b-0998-42c8-80ba-1c087e86e146\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"
Jul 25 01:32:57.524147 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.524087    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"
Jul 25 01:32:57.652862 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:57.652835    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"]
Jul 25 01:32:57.655434 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:32:57.655402    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d43c4b_0998_42c8_80ba_1c087e86e146.slice/crio-e254103daa549b7cf7ec466d639a576fa63e045df7979239d99bd8876e3d4caa WatchSource:0}: Error finding container e254103daa549b7cf7ec466d639a576fa63e045df7979239d99bd8876e3d4caa: Status 404 returned error can't find the container with id e254103daa549b7cf7ec466d639a576fa63e045df7979239d99bd8876e3d4caa
Jul 25 01:32:58.275838 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:58.275806    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z" event={"ID":"36d43c4b-0998-42c8-80ba-1c087e86e146","Type":"ContainerStarted","Data":"3cbbd8e005762edc8a25241b780e357ab0a143fbbccf37344d2b9220b5644d25"}
Jul 25 01:32:58.276197 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:58.275855    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z" event={"ID":"36d43c4b-0998-42c8-80ba-1c087e86e146","Type":"ContainerStarted","Data":"e254103daa549b7cf7ec466d639a576fa63e045df7979239d99bd8876e3d4caa"}
Jul 25 01:32:58.289924 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:58.289874    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z" podStartSLOduration=1.289858937 podStartE2EDuration="1.289858937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:32:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:32:58.288570744 +0000 UTC m=+3400.677093146" watchObservedRunningTime="2026-07-25 01:32:58.289858937 +0000 UTC m=+3400.678381339"
Jul 25 01:32:59.282077 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:59.282027    2566 generic.go:356] "Generic (PLEG): container finished" podID="36d43c4b-0998-42c8-80ba-1c087e86e146" containerID="3cbbd8e005762edc8a25241b780e357ab0a143fbbccf37344d2b9220b5644d25" exitCode=0
Jul 25 01:32:59.282479 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:32:59.282122    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z" event={"ID":"36d43c4b-0998-42c8-80ba-1c087e86e146","Type":"ContainerDied","Data":"3cbbd8e005762edc8a25241b780e357ab0a143fbbccf37344d2b9220b5644d25"}
Jul 25 01:33:00.406339 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:00.406317    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"
Jul 25 01:33:00.530469 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:00.530446    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d43c4b-0998-42c8-80ba-1c087e86e146-maas-api-service-ca\") pod \"36d43c4b-0998-42c8-80ba-1c087e86e146\" (UID: \"36d43c4b-0998-42c8-80ba-1c087e86e146\") "
Jul 25 01:33:00.530765 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:00.530743    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36d43c4b-0998-42c8-80ba-1c087e86e146-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "36d43c4b-0998-42c8-80ba-1c087e86e146" (UID: "36d43c4b-0998-42c8-80ba-1c087e86e146"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:33:00.631368 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:00.631342    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d43c4b-0998-42c8-80ba-1c087e86e146-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:33:01.292792 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:01.292766    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"
Jul 25 01:33:01.292926 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:01.292764    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z" event={"ID":"36d43c4b-0998-42c8-80ba-1c087e86e146","Type":"ContainerDied","Data":"e254103daa549b7cf7ec466d639a576fa63e045df7979239d99bd8876e3d4caa"}
Jul 25 01:33:01.292926 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:01.292910    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e254103daa549b7cf7ec466d639a576fa63e045df7979239d99bd8876e3d4caa"
Jul 25 01:33:11.593236 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:11.593151    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:33:12.741552 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:12.741512    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"]
Jul 25 01:33:12.742024 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:12.741803    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45" podUID="12ca0984-42f8-4e80-97ad-39707111c040" containerName="payload-processing" containerID="cri-o://a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199" gracePeriod=30
Jul 25 01:33:12.748090 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:12.748042    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"]
Jul 25 01:33:12.748638 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:12.748575    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2" podUID="eccd602a-24d2-4069-b102-65e90f7424ea" containerName="payload-pre-processing" containerID="cri-o://8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86" gracePeriod=30
Jul 25 01:33:13.004626 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.004597    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:33:13.014956 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.014937    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:33:13.127632 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.127606    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eccd602a-24d2-4069-b102-65e90f7424ea-plugins-config-volume\") pod \"eccd602a-24d2-4069-b102-65e90f7424ea\" (UID: \"eccd602a-24d2-4069-b102-65e90f7424ea\") "
Jul 25 01:33:13.127756 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.127640    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ca0984-42f8-4e80-97ad-39707111c040-plugins-config-volume\") pod \"12ca0984-42f8-4e80-97ad-39707111c040\" (UID: \"12ca0984-42f8-4e80-97ad-39707111c040\") "
Jul 25 01:33:13.127756 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.127677    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnzhk\" (UniqueName: \"kubernetes.io/projected/12ca0984-42f8-4e80-97ad-39707111c040-kube-api-access-jnzhk\") pod \"12ca0984-42f8-4e80-97ad-39707111c040\" (UID: \"12ca0984-42f8-4e80-97ad-39707111c040\") "
Jul 25 01:33:13.127844 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.127753    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rbjs\" (UniqueName: \"kubernetes.io/projected/eccd602a-24d2-4069-b102-65e90f7424ea-kube-api-access-7rbjs\") pod \"eccd602a-24d2-4069-b102-65e90f7424ea\" (UID: \"eccd602a-24d2-4069-b102-65e90f7424ea\") "
Jul 25 01:33:13.127988 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.127961    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12ca0984-42f8-4e80-97ad-39707111c040-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12ca0984-42f8-4e80-97ad-39707111c040" (UID: "12ca0984-42f8-4e80-97ad-39707111c040"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:33:13.128112 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.127997    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eccd602a-24d2-4069-b102-65e90f7424ea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eccd602a-24d2-4069-b102-65e90f7424ea" (UID: "eccd602a-24d2-4069-b102-65e90f7424ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:33:13.129661 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.129643    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ca0984-42f8-4e80-97ad-39707111c040-kube-api-access-jnzhk" (OuterVolumeSpecName: "kube-api-access-jnzhk") pod "12ca0984-42f8-4e80-97ad-39707111c040" (UID: "12ca0984-42f8-4e80-97ad-39707111c040"). InnerVolumeSpecName "kube-api-access-jnzhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:33:13.129722 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.129660    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eccd602a-24d2-4069-b102-65e90f7424ea-kube-api-access-7rbjs" (OuterVolumeSpecName: "kube-api-access-7rbjs") pod "eccd602a-24d2-4069-b102-65e90f7424ea" (UID: "eccd602a-24d2-4069-b102-65e90f7424ea"). InnerVolumeSpecName "kube-api-access-7rbjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:33:13.228489 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.228463    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnzhk\" (UniqueName: \"kubernetes.io/projected/12ca0984-42f8-4e80-97ad-39707111c040-kube-api-access-jnzhk\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:33:13.228489 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.228485    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rbjs\" (UniqueName: \"kubernetes.io/projected/eccd602a-24d2-4069-b102-65e90f7424ea-kube-api-access-7rbjs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:33:13.228623 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.228494    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eccd602a-24d2-4069-b102-65e90f7424ea-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:33:13.228623 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.228502    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ca0984-42f8-4e80-97ad-39707111c040-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:33:13.355613 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.355491    2566 generic.go:356] "Generic (PLEG): container finished" podID="eccd602a-24d2-4069-b102-65e90f7424ea" containerID="8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86" exitCode=0
Jul 25 01:33:13.355613 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.355543    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"
Jul 25 01:33:13.355613 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.355584    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2" event={"ID":"eccd602a-24d2-4069-b102-65e90f7424ea","Type":"ContainerDied","Data":"8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86"}
Jul 25 01:33:13.355877 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.355638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2" event={"ID":"eccd602a-24d2-4069-b102-65e90f7424ea","Type":"ContainerDied","Data":"8955f0323f62f8384c86ba02e170665de233ba4a3d752f04a23d3f33ff720b16"}
Jul 25 01:33:13.355877 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.355656    2566 scope.go:117] "RemoveContainer" containerID="8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86"
Jul 25 01:33:13.357437 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.357414    2566 generic.go:356] "Generic (PLEG): container finished" podID="12ca0984-42f8-4e80-97ad-39707111c040" containerID="a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199" exitCode=0
Jul 25 01:33:13.357536 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.357456    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"
Jul 25 01:33:13.357593 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.357450    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45" event={"ID":"12ca0984-42f8-4e80-97ad-39707111c040","Type":"ContainerDied","Data":"a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199"}
Jul 25 01:33:13.357632 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.357603    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45" event={"ID":"12ca0984-42f8-4e80-97ad-39707111c040","Type":"ContainerDied","Data":"0822fbdfcc8549f66cf95aa40c60738b11e9740abac95075a016a32e6426b36a"}
Jul 25 01:33:13.364881 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.364864    2566 scope.go:117] "RemoveContainer" containerID="8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86"
Jul 25 01:33:13.365397 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:33:13.365365    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86\": container with ID starting with 8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86 not found: ID does not exist" containerID="8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86"
Jul 25 01:33:13.365507 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.365404    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86"} err="failed to get container status \"8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86\": rpc error: code = NotFound desc = could not find container \"8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86\": container with ID starting with 8e926df57fcf945356577f9acddb9b8ea9d3d4856589b4ff56a913d2cf689f86 not found: ID does not exist"
Jul 25 01:33:13.365507 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.365425    2566 scope.go:117] "RemoveContainer" containerID="a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199"
Jul 25 01:33:13.373877 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.373856    2566 scope.go:117] "RemoveContainer" containerID="a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199"
Jul 25 01:33:13.374184 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:33:13.374162    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199\": container with ID starting with a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199 not found: ID does not exist" containerID="a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199"
Jul 25 01:33:13.374270 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.374190    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199"} err="failed to get container status \"a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199\": rpc error: code = NotFound desc = could not find container \"a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199\": container with ID starting with a3b5f669867c7d15f6b022651c169281f832ed706270d91b4c1aee4b9b68c199 not found: ID does not exist"
Jul 25 01:33:13.379921 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.379896    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"]
Jul 25 01:33:13.383507 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.383484    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7b2fa1-85c4d784c6-sqg45"]
Jul 25 01:33:13.393017 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.392996    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"]
Jul 25 01:33:13.396339 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:13.396318    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7b2fa1-9df4ddf88-nwnx2"]
Jul 25 01:33:14.171936 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:14.171905    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12ca0984-42f8-4e80-97ad-39707111c040" path="/var/lib/kubelet/pods/12ca0984-42f8-4e80-97ad-39707111c040/volumes"
Jul 25 01:33:14.172313 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:14.172294    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eccd602a-24d2-4069-b102-65e90f7424ea" path="/var/lib/kubelet/pods/eccd602a-24d2-4069-b102-65e90f7424ea/volumes"
Jul 25 01:33:17.286872 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:17.286836    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"]
Jul 25 01:33:17.290034 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:17.290006    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7b2fa1-dsv6z"]
Jul 25 01:33:18.171284 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:18.171249    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36d43c4b-0998-42c8-80ba-1c087e86e146" path="/var/lib/kubelet/pods/36d43c4b-0998-42c8-80ba-1c087e86e146/volumes"
Jul 25 01:33:22.289427 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:33:22.289391    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:34:05.910470 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.910429    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"]
Jul 25 01:34:05.910974 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.910784    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eccd602a-24d2-4069-b102-65e90f7424ea" containerName="payload-pre-processing"
Jul 25 01:34:05.910974 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.910796    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="eccd602a-24d2-4069-b102-65e90f7424ea" containerName="payload-pre-processing"
Jul 25 01:34:05.910974 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.910818    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36d43c4b-0998-42c8-80ba-1c087e86e146" containerName="revoke-keys"
Jul 25 01:34:05.910974 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.910824    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36d43c4b-0998-42c8-80ba-1c087e86e146" containerName="revoke-keys"
Jul 25 01:34:05.910974 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.910833    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12ca0984-42f8-4e80-97ad-39707111c040" containerName="payload-processing"
Jul 25 01:34:05.910974 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.910838    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="12ca0984-42f8-4e80-97ad-39707111c040" containerName="payload-processing"
Jul 25 01:34:05.910974 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.910892    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="12ca0984-42f8-4e80-97ad-39707111c040" containerName="payload-processing"
Jul 25 01:34:05.910974 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.910902    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="eccd602a-24d2-4069-b102-65e90f7424ea" containerName="payload-pre-processing"
Jul 25 01:34:05.910974 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.910908    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="36d43c4b-0998-42c8-80ba-1c087e86e146" containerName="revoke-keys"
Jul 25 01:34:05.914090 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.914055    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"
Jul 25 01:34:05.916328 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.916304    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:34:05.916473 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.916307    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:34:05.920473 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.920446    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"]
Jul 25 01:34:05.996285 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:05.996256    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12e9f00f-1229-4500-b296-b88cc13740de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2\" (UID: \"12e9f00f-1229-4500-b296-b88cc13740de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"
Jul 25 01:34:06.096551 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:06.096524    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12e9f00f-1229-4500-b296-b88cc13740de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2\" (UID: \"12e9f00f-1229-4500-b296-b88cc13740de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"
Jul 25 01:34:06.097117 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:06.097097    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12e9f00f-1229-4500-b296-b88cc13740de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2\" (UID: \"12e9f00f-1229-4500-b296-b88cc13740de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"
Jul 25 01:34:06.226715 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:06.226658    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"
Jul 25 01:34:06.356760 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:06.356736    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"]
Jul 25 01:34:06.358714 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:34:06.358686    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12e9f00f_1229_4500_b296_b88cc13740de.slice/crio-3065172513d67e3e72e179511afc8d8bf63ecf5244ca9c914bf5ddb0984a13d5 WatchSource:0}: Error finding container 3065172513d67e3e72e179511afc8d8bf63ecf5244ca9c914bf5ddb0984a13d5: Status 404 returned error can't find the container with id 3065172513d67e3e72e179511afc8d8bf63ecf5244ca9c914bf5ddb0984a13d5
Jul 25 01:34:06.600869 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:06.600840    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2" event={"ID":"12e9f00f-1229-4500-b296-b88cc13740de","Type":"ContainerStarted","Data":"189b3489a7e683ce54a1025ba2590146dff08e20b20ca7038c9f78cc64756844"}
Jul 25 01:34:06.600987 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:06.600925    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2" event={"ID":"12e9f00f-1229-4500-b296-b88cc13740de","Type":"ContainerStarted","Data":"3065172513d67e3e72e179511afc8d8bf63ecf5244ca9c914bf5ddb0984a13d5"}
Jul 25 01:34:06.615968 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:06.615923    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2" podStartSLOduration=1.615910822 podStartE2EDuration="1.615910822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:34:06.613943566 +0000 UTC m=+3469.002465970" watchObservedRunningTime="2026-07-25 01:34:06.615910822 +0000 UTC m=+3469.004433225"
Jul 25 01:34:07.606954 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:07.606926    2566 generic.go:356] "Generic (PLEG): container finished" podID="12e9f00f-1229-4500-b296-b88cc13740de" containerID="189b3489a7e683ce54a1025ba2590146dff08e20b20ca7038c9f78cc64756844" exitCode=0
Jul 25 01:34:07.607333 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:07.607022    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2" event={"ID":"12e9f00f-1229-4500-b296-b88cc13740de","Type":"ContainerDied","Data":"189b3489a7e683ce54a1025ba2590146dff08e20b20ca7038c9f78cc64756844"}
Jul 25 01:34:08.738717 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:08.738697    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"
Jul 25 01:34:08.811020 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:08.810996    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12e9f00f-1229-4500-b296-b88cc13740de-maas-api-service-ca\") pod \"12e9f00f-1229-4500-b296-b88cc13740de\" (UID: \"12e9f00f-1229-4500-b296-b88cc13740de\") "
Jul 25 01:34:08.811347 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:08.811327    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12e9f00f-1229-4500-b296-b88cc13740de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12e9f00f-1229-4500-b296-b88cc13740de" (UID: "12e9f00f-1229-4500-b296-b88cc13740de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:08.912128 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:08.912105    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12e9f00f-1229-4500-b296-b88cc13740de-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:09.617141 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:09.617105    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"
Jul 25 01:34:09.617287 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:09.617109    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2" event={"ID":"12e9f00f-1229-4500-b296-b88cc13740de","Type":"ContainerDied","Data":"3065172513d67e3e72e179511afc8d8bf63ecf5244ca9c914bf5ddb0984a13d5"}
Jul 25 01:34:09.617287 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:09.617250    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3065172513d67e3e72e179511afc8d8bf63ecf5244ca9c914bf5ddb0984a13d5"
Jul 25 01:34:16.424417 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.424383    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"]
Jul 25 01:34:16.424969 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.424641    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg" podUID="9a389fb8-76d3-4453-9a13-01dbf5ef4e28" containerName="payload-processing" containerID="cri-o://cc54bbe153a5448e8b65491fdc9609ac7d4301c785b8c0d98ae27c6ad7acc9c1" gracePeriod=30
Jul 25 01:34:16.429719 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.429693    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"]
Jul 25 01:34:16.429977 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.429935    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp" podUID="171f0eb2-9bbe-4b19-afc4-a223b6789316" containerName="payload-pre-processing" containerID="cri-o://8cbf3b316f4c1ea814bb6934fc78eee8364cdaa30588296cbc781866e2cc6567" gracePeriod=30
Jul 25 01:34:16.658224 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.658193    2566 generic.go:356] "Generic (PLEG): container finished" podID="171f0eb2-9bbe-4b19-afc4-a223b6789316" containerID="8cbf3b316f4c1ea814bb6934fc78eee8364cdaa30588296cbc781866e2cc6567" exitCode=0
Jul 25 01:34:16.658380 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.658270    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp" event={"ID":"171f0eb2-9bbe-4b19-afc4-a223b6789316","Type":"ContainerDied","Data":"8cbf3b316f4c1ea814bb6934fc78eee8364cdaa30588296cbc781866e2cc6567"}
Jul 25 01:34:16.659833 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.659807    2566 generic.go:356] "Generic (PLEG): container finished" podID="9a389fb8-76d3-4453-9a13-01dbf5ef4e28" containerID="cc54bbe153a5448e8b65491fdc9609ac7d4301c785b8c0d98ae27c6ad7acc9c1" exitCode=0
Jul 25 01:34:16.659943 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.659844    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg" event={"ID":"9a389fb8-76d3-4453-9a13-01dbf5ef4e28","Type":"ContainerDied","Data":"cc54bbe153a5448e8b65491fdc9609ac7d4301c785b8c0d98ae27c6ad7acc9c1"}
Jul 25 01:34:16.700877 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.700858    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:34:16.703514 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.703496    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:34:16.769973 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.769948    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ph7h\" (UniqueName: \"kubernetes.io/projected/171f0eb2-9bbe-4b19-afc4-a223b6789316-kube-api-access-6ph7h\") pod \"171f0eb2-9bbe-4b19-afc4-a223b6789316\" (UID: \"171f0eb2-9bbe-4b19-afc4-a223b6789316\") "
Jul 25 01:34:16.770103 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.769977    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkb5l\" (UniqueName: \"kubernetes.io/projected/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-kube-api-access-tkb5l\") pod \"9a389fb8-76d3-4453-9a13-01dbf5ef4e28\" (UID: \"9a389fb8-76d3-4453-9a13-01dbf5ef4e28\") "
Jul 25 01:34:16.770103 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.769998    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-plugins-config-volume\") pod \"9a389fb8-76d3-4453-9a13-01dbf5ef4e28\" (UID: \"9a389fb8-76d3-4453-9a13-01dbf5ef4e28\") "
Jul 25 01:34:16.770103 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.770038    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/171f0eb2-9bbe-4b19-afc4-a223b6789316-plugins-config-volume\") pod \"171f0eb2-9bbe-4b19-afc4-a223b6789316\" (UID: \"171f0eb2-9bbe-4b19-afc4-a223b6789316\") "
Jul 25 01:34:16.770424 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.770402    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9a389fb8-76d3-4453-9a13-01dbf5ef4e28" (UID: "9a389fb8-76d3-4453-9a13-01dbf5ef4e28"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:16.770488 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.770438    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/171f0eb2-9bbe-4b19-afc4-a223b6789316-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "171f0eb2-9bbe-4b19-afc4-a223b6789316" (UID: "171f0eb2-9bbe-4b19-afc4-a223b6789316"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:16.771955 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.771925    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/171f0eb2-9bbe-4b19-afc4-a223b6789316-kube-api-access-6ph7h" (OuterVolumeSpecName: "kube-api-access-6ph7h") pod "171f0eb2-9bbe-4b19-afc4-a223b6789316" (UID: "171f0eb2-9bbe-4b19-afc4-a223b6789316"). InnerVolumeSpecName "kube-api-access-6ph7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:34:16.772041 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.771995    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-kube-api-access-tkb5l" (OuterVolumeSpecName: "kube-api-access-tkb5l") pod "9a389fb8-76d3-4453-9a13-01dbf5ef4e28" (UID: "9a389fb8-76d3-4453-9a13-01dbf5ef4e28"). InnerVolumeSpecName "kube-api-access-tkb5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:34:16.870711 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.870690    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:16.870711 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.870708    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/171f0eb2-9bbe-4b19-afc4-a223b6789316-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:16.870825 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.870717    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ph7h\" (UniqueName: \"kubernetes.io/projected/171f0eb2-9bbe-4b19-afc4-a223b6789316-kube-api-access-6ph7h\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:16.870825 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:16.870726    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkb5l\" (UniqueName: \"kubernetes.io/projected/9a389fb8-76d3-4453-9a13-01dbf5ef4e28-kube-api-access-tkb5l\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:17.666291 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:17.666266    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"
Jul 25 01:34:17.666668 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:17.666304    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg" event={"ID":"9a389fb8-76d3-4453-9a13-01dbf5ef4e28","Type":"ContainerDied","Data":"8d612aa8dd5bf3c7fadceb9efd2a6ce0ec70500467e9befc9f67b82c2a047721"}
Jul 25 01:34:17.666668 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:17.666374    2566 scope.go:117] "RemoveContainer" containerID="cc54bbe153a5448e8b65491fdc9609ac7d4301c785b8c0d98ae27c6ad7acc9c1"
Jul 25 01:34:17.667846 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:17.667823    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp" event={"ID":"171f0eb2-9bbe-4b19-afc4-a223b6789316","Type":"ContainerDied","Data":"c4ebf2ebf2b0291bddacf85535819f7004f131738c46df6b96dd44c2789a5ba1"}
Jul 25 01:34:17.667929 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:17.667866    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"
Jul 25 01:34:17.676013 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:17.675899    2566 scope.go:117] "RemoveContainer" containerID="8cbf3b316f4c1ea814bb6934fc78eee8364cdaa30588296cbc781866e2cc6567"
Jul 25 01:34:17.691358 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:17.691304    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"]
Jul 25 01:34:17.694756 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:17.694732    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6bf6cd-7447f9454b-fqqwg"]
Jul 25 01:34:17.703811 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:17.703784    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"]
Jul 25 01:34:17.707648 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:17.707625    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6bf6cd-79bbb8b7dd-ttcmp"]
Jul 25 01:34:18.176657 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:18.176629    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="171f0eb2-9bbe-4b19-afc4-a223b6789316" path="/var/lib/kubelet/pods/171f0eb2-9bbe-4b19-afc4-a223b6789316/volumes"
Jul 25 01:34:18.176981 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:18.176967    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a389fb8-76d3-4453-9a13-01dbf5ef4e28" path="/var/lib/kubelet/pods/9a389fb8-76d3-4453-9a13-01dbf5ef4e28/volumes"
Jul 25 01:34:19.701210 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:19.701175    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:34:20.980398 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:20.980354    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"]
Jul 25 01:34:20.982452 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:20.982419    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6bf6cd-mpfx2"]
Jul 25 01:34:22.171154 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:22.171122    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12e9f00f-1229-4500-b296-b88cc13740de" path="/var/lib/kubelet/pods/12e9f00f-1229-4500-b296-b88cc13740de/volumes"
Jul 25 01:34:27.642495 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.642443    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"]
Jul 25 01:34:27.643018 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.642941    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="171f0eb2-9bbe-4b19-afc4-a223b6789316" containerName="payload-pre-processing"
Jul 25 01:34:27.643018 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.642963    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="171f0eb2-9bbe-4b19-afc4-a223b6789316" containerName="payload-pre-processing"
Jul 25 01:34:27.643018 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.642974    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12e9f00f-1229-4500-b296-b88cc13740de" containerName="revoke-keys"
Jul 25 01:34:27.643018 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.642981    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="12e9f00f-1229-4500-b296-b88cc13740de" containerName="revoke-keys"
Jul 25 01:34:27.643018 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.643002    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a389fb8-76d3-4453-9a13-01dbf5ef4e28" containerName="payload-processing"
Jul 25 01:34:27.643018 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.643011    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a389fb8-76d3-4453-9a13-01dbf5ef4e28" containerName="payload-processing"
Jul 25 01:34:27.643358 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.643135    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="171f0eb2-9bbe-4b19-afc4-a223b6789316" containerName="payload-pre-processing"
Jul 25 01:34:27.643358 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.643149    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a389fb8-76d3-4453-9a13-01dbf5ef4e28" containerName="payload-processing"
Jul 25 01:34:27.643358 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.643162    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="12e9f00f-1229-4500-b296-b88cc13740de" containerName="revoke-keys"
Jul 25 01:34:27.648279 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.648252    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"
Jul 25 01:34:27.650498 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.650472    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:34:27.650749 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.650546    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:34:27.651659 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.651630    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"]
Jul 25 01:34:27.747810 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.747788    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e28166-c3d5-4437-b51a-95f11a9fc432-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh\" (UID: \"54e28166-c3d5-4437-b51a-95f11a9fc432\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"
Jul 25 01:34:27.848137 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.848108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e28166-c3d5-4437-b51a-95f11a9fc432-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh\" (UID: \"54e28166-c3d5-4437-b51a-95f11a9fc432\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"
Jul 25 01:34:27.848713 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.848690    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e28166-c3d5-4437-b51a-95f11a9fc432-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh\" (UID: \"54e28166-c3d5-4437-b51a-95f11a9fc432\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"
Jul 25 01:34:27.962419 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:27.962359    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"
Jul 25 01:34:28.294193 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:28.294169    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"]
Jul 25 01:34:28.296288 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:34:28.296256    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54e28166_c3d5_4437_b51a_95f11a9fc432.slice/crio-877a0b9901f99fc3b4b541a3656b93244c6f807d9c3088600b14b9a5012aac70 WatchSource:0}: Error finding container 877a0b9901f99fc3b4b541a3656b93244c6f807d9c3088600b14b9a5012aac70: Status 404 returned error can't find the container with id 877a0b9901f99fc3b4b541a3656b93244c6f807d9c3088600b14b9a5012aac70
Jul 25 01:34:28.714750 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:28.714711    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh" event={"ID":"54e28166-c3d5-4437-b51a-95f11a9fc432","Type":"ContainerStarted","Data":"ab43cfd6c7cfca2260c58b418894b7459533d1987a324a5858ba1f68a3ff5eee"}
Jul 25 01:34:28.715082 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:28.714772    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh" event={"ID":"54e28166-c3d5-4437-b51a-95f11a9fc432","Type":"ContainerStarted","Data":"877a0b9901f99fc3b4b541a3656b93244c6f807d9c3088600b14b9a5012aac70"}
Jul 25 01:34:28.729426 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:28.729378    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh" podStartSLOduration=1.729358306 podStartE2EDuration="1.729358306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:34:28.72925736 +0000 UTC m=+3491.117779762" watchObservedRunningTime="2026-07-25 01:34:28.729358306 +0000 UTC m=+3491.117880711"
Jul 25 01:34:29.720957 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:29.720923    2566 generic.go:356] "Generic (PLEG): container finished" podID="54e28166-c3d5-4437-b51a-95f11a9fc432" containerID="ab43cfd6c7cfca2260c58b418894b7459533d1987a324a5858ba1f68a3ff5eee" exitCode=0
Jul 25 01:34:29.721333 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:29.721003    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh" event={"ID":"54e28166-c3d5-4437-b51a-95f11a9fc432","Type":"ContainerDied","Data":"ab43cfd6c7cfca2260c58b418894b7459533d1987a324a5858ba1f68a3ff5eee"}
Jul 25 01:34:30.878652 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:30.878617    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"
Jul 25 01:34:30.973033 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:30.973004    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e28166-c3d5-4437-b51a-95f11a9fc432-maas-api-service-ca\") pod \"54e28166-c3d5-4437-b51a-95f11a9fc432\" (UID: \"54e28166-c3d5-4437-b51a-95f11a9fc432\") "
Jul 25 01:34:30.973336 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:30.973312    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54e28166-c3d5-4437-b51a-95f11a9fc432-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54e28166-c3d5-4437-b51a-95f11a9fc432" (UID: "54e28166-c3d5-4437-b51a-95f11a9fc432"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:31.073696 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:31.073634    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54e28166-c3d5-4437-b51a-95f11a9fc432-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:31.730989 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:31.730967    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"
Jul 25 01:34:31.731127 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:31.731012    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh" event={"ID":"54e28166-c3d5-4437-b51a-95f11a9fc432","Type":"ContainerDied","Data":"877a0b9901f99fc3b4b541a3656b93244c6f807d9c3088600b14b9a5012aac70"}
Jul 25 01:34:31.731127 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:31.731083    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="877a0b9901f99fc3b4b541a3656b93244c6f807d9c3088600b14b9a5012aac70"
Jul 25 01:34:38.051141 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.050427    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"]
Jul 25 01:34:38.052276 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.051999    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2" podUID="292eeca0-d800-4fb0-9d3b-6aa4763b736e" containerName="payload-processing" containerID="cri-o://dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04" gracePeriod=30
Jul 25 01:34:38.053150 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.053056    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"]
Jul 25 01:34:38.053380 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.053307    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h" podUID="9252a56a-fec6-4100-b0c4-bdafa9a0ab29" containerName="payload-pre-processing" containerID="cri-o://e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800" gracePeriod=30
Jul 25 01:34:38.122441 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.122405    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h" podUID="9252a56a-fec6-4100-b0c4-bdafa9a0ab29" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.42:9004: connect: connection refused"
Jul 25 01:34:38.122536 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.122410    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2" podUID="292eeca0-d800-4fb0-9d3b-6aa4763b736e" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.43:9004: connect: connection refused"
Jul 25 01:34:38.310337 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.310315    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:34:38.318743 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.318724    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:34:38.424156 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.424126    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-plugins-config-volume\") pod \"9252a56a-fec6-4100-b0c4-bdafa9a0ab29\" (UID: \"9252a56a-fec6-4100-b0c4-bdafa9a0ab29\") "
Jul 25 01:34:38.424274 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.424184    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/292eeca0-d800-4fb0-9d3b-6aa4763b736e-plugins-config-volume\") pod \"292eeca0-d800-4fb0-9d3b-6aa4763b736e\" (UID: \"292eeca0-d800-4fb0-9d3b-6aa4763b736e\") "
Jul 25 01:34:38.424274 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.424248    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldj5b\" (UniqueName: \"kubernetes.io/projected/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-kube-api-access-ldj5b\") pod \"9252a56a-fec6-4100-b0c4-bdafa9a0ab29\" (UID: \"9252a56a-fec6-4100-b0c4-bdafa9a0ab29\") "
Jul 25 01:34:38.424274 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.424269    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5vw5\" (UniqueName: \"kubernetes.io/projected/292eeca0-d800-4fb0-9d3b-6aa4763b736e-kube-api-access-z5vw5\") pod \"292eeca0-d800-4fb0-9d3b-6aa4763b736e\" (UID: \"292eeca0-d800-4fb0-9d3b-6aa4763b736e\") "
Jul 25 01:34:38.424539 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.424510    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9252a56a-fec6-4100-b0c4-bdafa9a0ab29" (UID: "9252a56a-fec6-4100-b0c4-bdafa9a0ab29"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:38.424669 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.424588    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/292eeca0-d800-4fb0-9d3b-6aa4763b736e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "292eeca0-d800-4fb0-9d3b-6aa4763b736e" (UID: "292eeca0-d800-4fb0-9d3b-6aa4763b736e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:38.426397 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.426375    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/292eeca0-d800-4fb0-9d3b-6aa4763b736e-kube-api-access-z5vw5" (OuterVolumeSpecName: "kube-api-access-z5vw5") pod "292eeca0-d800-4fb0-9d3b-6aa4763b736e" (UID: "292eeca0-d800-4fb0-9d3b-6aa4763b736e"). InnerVolumeSpecName "kube-api-access-z5vw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:34:38.426467 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.426425    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-kube-api-access-ldj5b" (OuterVolumeSpecName: "kube-api-access-ldj5b") pod "9252a56a-fec6-4100-b0c4-bdafa9a0ab29" (UID: "9252a56a-fec6-4100-b0c4-bdafa9a0ab29"). InnerVolumeSpecName "kube-api-access-ldj5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:34:38.525101 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.525058    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldj5b\" (UniqueName: \"kubernetes.io/projected/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-kube-api-access-ldj5b\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:38.525101 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.525103    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5vw5\" (UniqueName: \"kubernetes.io/projected/292eeca0-d800-4fb0-9d3b-6aa4763b736e-kube-api-access-z5vw5\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:38.525236 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.525112    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9252a56a-fec6-4100-b0c4-bdafa9a0ab29-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:38.525236 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.525123    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/292eeca0-d800-4fb0-9d3b-6aa4763b736e-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:38.762887 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.762851    2566 generic.go:356] "Generic (PLEG): container finished" podID="292eeca0-d800-4fb0-9d3b-6aa4763b736e" containerID="dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04" exitCode=0
Jul 25 01:34:38.763025 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.762904    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"
Jul 25 01:34:38.763025 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.762944    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2" event={"ID":"292eeca0-d800-4fb0-9d3b-6aa4763b736e","Type":"ContainerDied","Data":"dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04"}
Jul 25 01:34:38.763025 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.763002    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2" event={"ID":"292eeca0-d800-4fb0-9d3b-6aa4763b736e","Type":"ContainerDied","Data":"9eede4a07a07df449853c87a98a25876eb1f5d7a73bf27b80e8b118f9812b322"}
Jul 25 01:34:38.763025 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.763019    2566 scope.go:117] "RemoveContainer" containerID="dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04"
Jul 25 01:34:38.764498 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.764471    2566 generic.go:356] "Generic (PLEG): container finished" podID="9252a56a-fec6-4100-b0c4-bdafa9a0ab29" containerID="e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800" exitCode=0
Jul 25 01:34:38.764612 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.764526    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h" event={"ID":"9252a56a-fec6-4100-b0c4-bdafa9a0ab29","Type":"ContainerDied","Data":"e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800"}
Jul 25 01:34:38.764612 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.764566    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h" event={"ID":"9252a56a-fec6-4100-b0c4-bdafa9a0ab29","Type":"ContainerDied","Data":"fe90723410cc05314cddf0f4d6e0a1147013db87123b8967487f66b691a86213"}
Jul 25 01:34:38.764612 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.764529    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"
Jul 25 01:34:38.773609 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.773587    2566 scope.go:117] "RemoveContainer" containerID="dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04"
Jul 25 01:34:38.773915 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:34:38.773886    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04\": container with ID starting with dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04 not found: ID does not exist" containerID="dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04"
Jul 25 01:34:38.774001 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.773927    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04"} err="failed to get container status \"dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04\": rpc error: code = NotFound desc = could not find container \"dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04\": container with ID starting with dcf0fa5ae7afe1f36f3ba957ca54566444894ec4bb4a6c31fcf0f1e868e44a04 not found: ID does not exist"
Jul 25 01:34:38.774001 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.773951    2566 scope.go:117] "RemoveContainer" containerID="e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800"
Jul 25 01:34:38.783905 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.783803    2566 scope.go:117] "RemoveContainer" containerID="e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800"
Jul 25 01:34:38.784623 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:34:38.784593    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800\": container with ID starting with e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800 not found: ID does not exist" containerID="e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800"
Jul 25 01:34:38.784719 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.784630    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800"} err="failed to get container status \"e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800\": rpc error: code = NotFound desc = could not find container \"e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800\": container with ID starting with e0fa7c1736a31ba363fc53b54236affc64a48442584d3fab29fd06553216f800 not found: ID does not exist"
Jul 25 01:34:38.787835 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.787806    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"]
Jul 25 01:34:38.791477 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.791453    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a1fe77-6d6fccc6cb-kzf4h"]
Jul 25 01:34:38.801355 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.801327    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"]
Jul 25 01:34:38.802927 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:38.802906    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a1fe77-549cbd4b58-ln8n2"]
Jul 25 01:34:40.172423 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:40.172358    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="292eeca0-d800-4fb0-9d3b-6aa4763b736e" path="/var/lib/kubelet/pods/292eeca0-d800-4fb0-9d3b-6aa4763b736e/volumes"
Jul 25 01:34:40.172757 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:40.172703    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9252a56a-fec6-4100-b0c4-bdafa9a0ab29" path="/var/lib/kubelet/pods/9252a56a-fec6-4100-b0c4-bdafa9a0ab29/volumes"
Jul 25 01:34:41.290182 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:41.290144    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:34:42.709689 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:42.709653    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"]
Jul 25 01:34:42.711433 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:42.711405    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1fe77-rnshh"]
Jul 25 01:34:44.171142 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:44.171105    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54e28166-c3d5-4437-b51a-95f11a9fc432" path="/var/lib/kubelet/pods/54e28166-c3d5-4437-b51a-95f11a9fc432/volumes"
Jul 25 01:34:49.670431 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.670394    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"]
Jul 25 01:34:49.670959 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.670701    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54e28166-c3d5-4437-b51a-95f11a9fc432" containerName="revoke-keys"
Jul 25 01:34:49.670959 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.670714    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="54e28166-c3d5-4437-b51a-95f11a9fc432" containerName="revoke-keys"
Jul 25 01:34:49.670959 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.670725    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9252a56a-fec6-4100-b0c4-bdafa9a0ab29" containerName="payload-pre-processing"
Jul 25 01:34:49.670959 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.670730    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9252a56a-fec6-4100-b0c4-bdafa9a0ab29" containerName="payload-pre-processing"
Jul 25 01:34:49.670959 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.670739    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="292eeca0-d800-4fb0-9d3b-6aa4763b736e" containerName="payload-processing"
Jul 25 01:34:49.670959 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.670745    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="292eeca0-d800-4fb0-9d3b-6aa4763b736e" containerName="payload-processing"
Jul 25 01:34:49.670959 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.670827    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="54e28166-c3d5-4437-b51a-95f11a9fc432" containerName="revoke-keys"
Jul 25 01:34:49.670959 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.670839    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="292eeca0-d800-4fb0-9d3b-6aa4763b736e" containerName="payload-processing"
Jul 25 01:34:49.670959 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.670845    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9252a56a-fec6-4100-b0c4-bdafa9a0ab29" containerName="payload-pre-processing"
Jul 25 01:34:49.674951 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.674918    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"
Jul 25 01:34:49.677281 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.677246    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:34:49.677586 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.677558    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 01:34:49.681363 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.681337    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"]
Jul 25 01:34:49.704018 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.703988    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee972729-15cc-4a11-90fd-3a61e95f292f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh\" (UID: \"ee972729-15cc-4a11-90fd-3a61e95f292f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"
Jul 25 01:34:49.805377 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.805343    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee972729-15cc-4a11-90fd-3a61e95f292f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh\" (UID: \"ee972729-15cc-4a11-90fd-3a61e95f292f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"
Jul 25 01:34:49.806006 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.805985    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee972729-15cc-4a11-90fd-3a61e95f292f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh\" (UID: \"ee972729-15cc-4a11-90fd-3a61e95f292f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"
Jul 25 01:34:49.986556 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:49.986477    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"
Jul 25 01:34:50.111094 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:50.111044    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"]
Jul 25 01:34:50.113781 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:34:50.113753    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee972729_15cc_4a11_90fd_3a61e95f292f.slice/crio-bf303363d657de7aadbf8d96cd1b0ab5e2e17468a6b3f9d109146c42eb421249 WatchSource:0}: Error finding container bf303363d657de7aadbf8d96cd1b0ab5e2e17468a6b3f9d109146c42eb421249: Status 404 returned error can't find the container with id bf303363d657de7aadbf8d96cd1b0ab5e2e17468a6b3f9d109146c42eb421249
Jul 25 01:34:50.829210 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:50.829172    2566 generic.go:356] "Generic (PLEG): container finished" podID="ee972729-15cc-4a11-90fd-3a61e95f292f" containerID="3afd50bc4837cd619b57ea376ae704e1cf827a631120eaf94b52a568171b2b41" exitCode=0
Jul 25 01:34:50.829555 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:50.829248    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh" event={"ID":"ee972729-15cc-4a11-90fd-3a61e95f292f","Type":"ContainerDied","Data":"3afd50bc4837cd619b57ea376ae704e1cf827a631120eaf94b52a568171b2b41"}
Jul 25 01:34:50.829555 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:50.829299    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh" event={"ID":"ee972729-15cc-4a11-90fd-3a61e95f292f","Type":"ContainerStarted","Data":"bf303363d657de7aadbf8d96cd1b0ab5e2e17468a6b3f9d109146c42eb421249"}
Jul 25 01:34:51.960390 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:51.960366    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"
Jul 25 01:34:52.020627 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:52.020597    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee972729-15cc-4a11-90fd-3a61e95f292f-maas-api-service-ca\") pod \"ee972729-15cc-4a11-90fd-3a61e95f292f\" (UID: \"ee972729-15cc-4a11-90fd-3a61e95f292f\") "
Jul 25 01:34:52.020922 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:52.020898    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee972729-15cc-4a11-90fd-3a61e95f292f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ee972729-15cc-4a11-90fd-3a61e95f292f" (UID: "ee972729-15cc-4a11-90fd-3a61e95f292f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:52.121299 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:52.121280    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee972729-15cc-4a11-90fd-3a61e95f292f-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:34:52.840871 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:52.840843    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh" event={"ID":"ee972729-15cc-4a11-90fd-3a61e95f292f","Type":"ContainerDied","Data":"bf303363d657de7aadbf8d96cd1b0ab5e2e17468a6b3f9d109146c42eb421249"}
Jul 25 01:34:52.840991 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:52.840876    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"
Jul 25 01:34:52.840991 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:34:52.840888    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf303363d657de7aadbf8d96cd1b0ab5e2e17468a6b3f9d109146c42eb421249"
Jul 25 01:35:00.462656 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.462618    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"]
Jul 25 01:35:00.463196 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.462798    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c" podUID="92e8bc11-65b6-4005-bcd4-ee3ab005745b" containerName="payload-processing" containerID="cri-o://d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472" gracePeriod=30
Jul 25 01:35:00.472970 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.472783    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"]
Jul 25 01:35:00.473100 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.472997    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm" podUID="e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2" containerName="payload-pre-processing" containerID="cri-o://2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff" gracePeriod=30
Jul 25 01:35:00.717338 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.717287    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:35:00.748741 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.748719    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:35:00.786784 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.786763    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmkt2\" (UniqueName: \"kubernetes.io/projected/92e8bc11-65b6-4005-bcd4-ee3ab005745b-kube-api-access-rmkt2\") pod \"92e8bc11-65b6-4005-bcd4-ee3ab005745b\" (UID: \"92e8bc11-65b6-4005-bcd4-ee3ab005745b\") "
Jul 25 01:35:00.786897 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.786827    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92e8bc11-65b6-4005-bcd4-ee3ab005745b-plugins-config-volume\") pod \"92e8bc11-65b6-4005-bcd4-ee3ab005745b\" (UID: \"92e8bc11-65b6-4005-bcd4-ee3ab005745b\") "
Jul 25 01:35:00.787199 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.787179    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92e8bc11-65b6-4005-bcd4-ee3ab005745b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "92e8bc11-65b6-4005-bcd4-ee3ab005745b" (UID: "92e8bc11-65b6-4005-bcd4-ee3ab005745b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:35:00.788708 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.788689    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92e8bc11-65b6-4005-bcd4-ee3ab005745b-kube-api-access-rmkt2" (OuterVolumeSpecName: "kube-api-access-rmkt2") pod "92e8bc11-65b6-4005-bcd4-ee3ab005745b" (UID: "92e8bc11-65b6-4005-bcd4-ee3ab005745b"). InnerVolumeSpecName "kube-api-access-rmkt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:35:00.875841 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.875815    2566 generic.go:356] "Generic (PLEG): container finished" podID="92e8bc11-65b6-4005-bcd4-ee3ab005745b" containerID="d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472" exitCode=0
Jul 25 01:35:00.875922 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.875853    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"
Jul 25 01:35:00.875922 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.875901    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c" event={"ID":"92e8bc11-65b6-4005-bcd4-ee3ab005745b","Type":"ContainerDied","Data":"d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472"}
Jul 25 01:35:00.876006 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.875953    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c" event={"ID":"92e8bc11-65b6-4005-bcd4-ee3ab005745b","Type":"ContainerDied","Data":"ac95385a2f9dd9c20438e099b8412ff4d98802d1e9cd771e74f8468c96109549"}
Jul 25 01:35:00.876006 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.875980    2566 scope.go:117] "RemoveContainer" containerID="d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472"
Jul 25 01:35:00.877187 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.877165    2566 generic.go:356] "Generic (PLEG): container finished" podID="e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2" containerID="2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff" exitCode=0
Jul 25 01:35:00.877267 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.877211    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"
Jul 25 01:35:00.877267 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.877245    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm" event={"ID":"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2","Type":"ContainerDied","Data":"2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff"}
Jul 25 01:35:00.877350 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.877286    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm" event={"ID":"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2","Type":"ContainerDied","Data":"3f9e2f5038c5fba1131400ec4cb9f94947227bd496d14e200e901fe78f16fb61"}
Jul 25 01:35:00.884472 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.884457    2566 scope.go:117] "RemoveContainer" containerID="d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472"
Jul 25 01:35:00.884702 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:35:00.884683    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472\": container with ID starting with d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472 not found: ID does not exist" containerID="d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472"
Jul 25 01:35:00.884748 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.884710    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472"} err="failed to get container status \"d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472\": rpc error: code = NotFound desc = could not find container \"d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472\": container with ID starting with d90136feeacce0d8eb8417fd9cb3f08c288595d0e9c15ae4aff796fc3f440472 not found: ID does not exist"
Jul 25 01:35:00.884748 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.884737    2566 scope.go:117] "RemoveContainer" containerID="2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff"
Jul 25 01:35:00.887800 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.887780    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cnx5\" (UniqueName: \"kubernetes.io/projected/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-kube-api-access-2cnx5\") pod \"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2\" (UID: \"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2\") "
Jul 25 01:35:00.887894 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.887874    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-plugins-config-volume\") pod \"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2\" (UID: \"e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2\") "
Jul 25 01:35:00.888108 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.888090    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92e8bc11-65b6-4005-bcd4-ee3ab005745b-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:35:00.888165 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.888110    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmkt2\" (UniqueName: \"kubernetes.io/projected/92e8bc11-65b6-4005-bcd4-ee3ab005745b-kube-api-access-rmkt2\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:35:00.888226 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.888207    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2" (UID: "e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:35:00.889941 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.889922    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-kube-api-access-2cnx5" (OuterVolumeSpecName: "kube-api-access-2cnx5") pod "e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2" (UID: "e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2"). InnerVolumeSpecName "kube-api-access-2cnx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:35:00.892143 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.892126    2566 scope.go:117] "RemoveContainer" containerID="2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff"
Jul 25 01:35:00.892383 ip-10-0-140-134 kubenswrapper[2566]: E0725 01:35:00.892366    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff\": container with ID starting with 2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff not found: ID does not exist" containerID="2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff"
Jul 25 01:35:00.892422 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.892390    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff"} err="failed to get container status \"2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff\": rpc error: code = NotFound desc = could not find container \"2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff\": container with ID starting with 2a965038b5ad0ffaaf250d812538fdc76e0f58074b459556c081c728496448ff not found: ID does not exist"
Jul 25 01:35:00.895376 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.895357    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"]
Jul 25 01:35:00.897334 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.897315    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d50ef4-54954656cd-k9v5c"]
Jul 25 01:35:00.988751 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.988695    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cnx5\" (UniqueName: \"kubernetes.io/projected/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-kube-api-access-2cnx5\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:35:00.988751 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:00.988720    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2-plugins-config-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 25 01:35:01.202838 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:01.202790    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"]
Jul 25 01:35:01.203103 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:01.202848    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d50ef4-747887c4d8-ztrlm"]
Jul 25 01:35:02.171193 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:02.171161    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92e8bc11-65b6-4005-bcd4-ee3ab005745b" path="/var/lib/kubelet/pods/92e8bc11-65b6-4005-bcd4-ee3ab005745b/volumes"
Jul 25 01:35:02.171535 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:02.171510    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2" path="/var/lib/kubelet/pods/e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2/volumes"
Jul 25 01:35:03.488517 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:03.488478    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqmmx"]
Jul 25 01:35:04.729636 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:04.729596    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"]
Jul 25 01:35:04.731427 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:04.731394    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d50ef4-fcpzh"]
Jul 25 01:35:06.171128 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:06.171093    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee972729-15cc-4a11-90fd-3a61e95f292f" path="/var/lib/kubelet/pods/ee972729-15cc-4a11-90fd-3a61e95f292f/volumes"
Jul 25 01:35:18.377020 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:18.376989    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mn5hf_250dc570-ac6d-45e0-956d-ffa2996315d1/manager/0.log"
Jul 25 01:35:18.647943 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:18.647883    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b8579fc9-dbl2k_b78e7779-41b6-4383-834a-1382d10f83f1/copy-manifests/0.log"
Jul 25 01:35:18.654938 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:18.654908    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b8579fc9-dbl2k_b78e7779-41b6-4383-834a-1382d10f83f1/manager/0.log"
Jul 25 01:35:21.848117 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:21.848087    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sqmmx_19385261-42c9-494d-8847-3e80717fe0c5/limitador/0.log"
Jul 25 01:35:22.512719 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:22.512683    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-np2wj_7d9483e9-b41a-477b-8e82-9c1310d22c01/discovery/0.log"
Jul 25 01:35:23.999844 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:23.999806    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmwmxs_46c22c91-e51b-4dcc-9b8e-ef7c245c6cce/main/0.log"
Jul 25 01:35:32.685994 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:32.685953    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8vm5n_c2cb88dd-222f-483e-8623-ead0a2448f6e/global-pull-secret-syncer/0.log"
Jul 25 01:35:32.908243 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:32.908216    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h8h45_663a5a14-c43d-4aa9-a799-03e303968608/konnectivity-agent/0.log"
Jul 25 01:35:32.949813 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:32.949757    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-134.ec2.internal_22634a54757c03b5d1c07ddcc38ba334/haproxy/0.log"
Jul 25 01:35:38.188747 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:38.188704    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sqmmx_19385261-42c9-494d-8847-3e80717fe0c5/limitador/0.log"
Jul 25 01:35:40.196881 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:40.196854    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7bb68b47f5-bgjg8_84ee46b2-c55d-49c1-b9e8-64d4532efc86/metrics-server/0.log"
Jul 25 01:35:40.473560 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:40.473504    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xrqns_86cb50a5-8a7f-4c7f-be32-8569e70be687/node-exporter/0.log"
Jul 25 01:35:40.504409 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:40.504388    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xrqns_86cb50a5-8a7f-4c7f-be32-8569e70be687/kube-rbac-proxy/0.log"
Jul 25 01:35:40.536150 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:40.536130    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xrqns_86cb50a5-8a7f-4c7f-be32-8569e70be687/init-textfile/0.log"
Jul 25 01:35:40.591388 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:40.591369    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-7kbx8_5a0e2850-6e87-4536-ac5a-1218d2f32244/kube-rbac-proxy-main/0.log"
Jul 25 01:35:40.628234 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:40.628212    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-7kbx8_5a0e2850-6e87-4536-ac5a-1218d2f32244/kube-rbac-proxy-self/0.log"
Jul 25 01:35:40.666025 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:40.666001    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-7kbx8_5a0e2850-6e87-4536-ac5a-1218d2f32244/openshift-state-metrics/0.log"
Jul 25 01:35:43.308268 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.308231    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"]
Jul 25 01:35:43.308676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.308539    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee972729-15cc-4a11-90fd-3a61e95f292f" containerName="revoke-keys"
Jul 25 01:35:43.308676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.308554    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee972729-15cc-4a11-90fd-3a61e95f292f" containerName="revoke-keys"
Jul 25 01:35:43.308676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.308564    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92e8bc11-65b6-4005-bcd4-ee3ab005745b" containerName="payload-processing"
Jul 25 01:35:43.308676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.308570    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="92e8bc11-65b6-4005-bcd4-ee3ab005745b" containerName="payload-processing"
Jul 25 01:35:43.308676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.308577    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2" containerName="payload-pre-processing"
Jul 25 01:35:43.308676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.308583    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2" containerName="payload-pre-processing"
Jul 25 01:35:43.308676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.308630    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4bded37-5da4-4e0a-81c6-a4fa62f9b8a2" containerName="payload-pre-processing"
Jul 25 01:35:43.308676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.308640    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee972729-15cc-4a11-90fd-3a61e95f292f" containerName="revoke-keys"
Jul 25 01:35:43.308676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.308647    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="92e8bc11-65b6-4005-bcd4-ee3ab005745b" containerName="payload-processing"
Jul 25 01:35:43.311542 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.311521    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.313554 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.313530    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bt87\"/\"kube-root-ca.crt\""
Jul 25 01:35:43.314294 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.314272    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bt87\"/\"openshift-service-ca.crt\""
Jul 25 01:35:43.314294 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.314272    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8bt87\"/\"default-dockercfg-jr6f2\""
Jul 25 01:35:43.318054 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.318022    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"]
Jul 25 01:35:43.483581 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.483547    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-sys\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.483701 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.483584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llgl4\" (UniqueName: \"kubernetes.io/projected/97be97c4-7a55-48f3-842d-a76865d7a11d-kube-api-access-llgl4\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.483701 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.483618    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-podres\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.483701 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.483687    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-lib-modules\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.483820 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.483778    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-proc\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.585056 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.584993    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-proc\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.585056 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.585032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-sys\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.585056 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.585051    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llgl4\" (UniqueName: \"kubernetes.io/projected/97be97c4-7a55-48f3-842d-a76865d7a11d-kube-api-access-llgl4\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.585244 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.585123    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-proc\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.585244 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.585142    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-sys\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.585244 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.585171    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-podres\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.585244 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.585202    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-lib-modules\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.585373 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.585313    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-podres\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.585373 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.585339    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97be97c4-7a55-48f3-842d-a76865d7a11d-lib-modules\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.592220 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.592195    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llgl4\" (UniqueName: \"kubernetes.io/projected/97be97c4-7a55-48f3-842d-a76865d7a11d-kube-api-access-llgl4\") pod \"perf-node-gather-daemonset-tnbqk\" (UID: \"97be97c4-7a55-48f3-842d-a76865d7a11d\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.622845 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.622822    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:43.747872 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:43.747822    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"]
Jul 25 01:35:43.750449 ip-10-0-140-134 kubenswrapper[2566]: W0725 01:35:43.750421    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod97be97c4_7a55_48f3_842d_a76865d7a11d.slice/crio-2ab5fe9f7ea8a286d4d9d03622441f17753374c7492b0903a439807546363bff WatchSource:0}: Error finding container 2ab5fe9f7ea8a286d4d9d03622441f17753374c7492b0903a439807546363bff: Status 404 returned error can't find the container with id 2ab5fe9f7ea8a286d4d9d03622441f17753374c7492b0903a439807546363bff
Jul 25 01:35:44.081745 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:44.081711    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk" event={"ID":"97be97c4-7a55-48f3-842d-a76865d7a11d","Type":"ContainerStarted","Data":"cf1ebe052c716713e777986b029d83f16b66e0ecc36d3c90f4e63814ab7d8714"}
Jul 25 01:35:44.081869 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:44.081768    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk" event={"ID":"97be97c4-7a55-48f3-842d-a76865d7a11d","Type":"ContainerStarted","Data":"2ab5fe9f7ea8a286d4d9d03622441f17753374c7492b0903a439807546363bff"}
Jul 25 01:35:44.081869 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:44.081807    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:44.096385 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:44.096336    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk" podStartSLOduration=1.096321291 podStartE2EDuration="1.096321291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:35:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:35:44.095138233 +0000 UTC m=+3566.483660636" watchObservedRunningTime="2026-07-25 01:35:44.096321291 +0000 UTC m=+3566.484843672"
Jul 25 01:35:45.486586 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:45.486554    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hw84r_e7839bc1-c826-40cd-9428-4e7080474366/dns/0.log"
Jul 25 01:35:45.506894 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:45.506874    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hw84r_e7839bc1-c826-40cd-9428-4e7080474366/kube-rbac-proxy/0.log"
Jul 25 01:35:45.625529 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:45.625505    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kpkfg_34cc6d1e-0b4a-420e-bd17-5b5fb36ba380/dns-node-resolver/0.log"
Jul 25 01:35:46.204095 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:46.204059    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dxmdc_8d1dc5e7-7792-4763-a69e-312272d69305/node-ca/0.log"
Jul 25 01:35:47.333196 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:47.333169    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-np2wj_7d9483e9-b41a-477b-8e82-9c1310d22c01/discovery/0.log"
Jul 25 01:35:48.335567 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:48.335541    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b9hmx_7de09553-7361-41fb-b3e3-2776190c44c7/serve-healthcheck-canary/0.log"
Jul 25 01:35:48.864676 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:48.864642    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6xwds_fc9f4877-ba9e-46f0-8716-0564f6b1f79b/kube-rbac-proxy/0.log"
Jul 25 01:35:48.884390 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:48.884364    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6xwds_fc9f4877-ba9e-46f0-8716-0564f6b1f79b/exporter/0.log"
Jul 25 01:35:48.903568 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:48.903548    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6xwds_fc9f4877-ba9e-46f0-8716-0564f6b1f79b/extractor/0.log"
Jul 25 01:35:50.096931 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:50.096905    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-tnbqk"
Jul 25 01:35:51.843049 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:51.843018    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mn5hf_250dc570-ac6d-45e0-956d-ffa2996315d1/manager/0.log"
Jul 25 01:35:51.912474 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:51.912448    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b8579fc9-dbl2k_b78e7779-41b6-4383-834a-1382d10f83f1/manager/0.log"
Jul 25 01:35:51.935596 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:51.935572    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b8579fc9-dbl2k_b78e7779-41b6-4383-834a-1382d10f83f1/copy-manifests/0.log"
Jul 25 01:35:53.604563 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:53.604500    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7bd5dcbf9d-bql5c_afc8c728-47f9-41f2-9181-c814c3f0ec52/manager/0.log"
Jul 25 01:35:53.633245 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:53.633227    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-pnx2s_d55a2a5f-a2cd-424c-8055-3627848c2cfe/openshift-lws-operator/0.log"
Jul 25 01:35:54.528563 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:54.528541    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-np2wj_7d9483e9-b41a-477b-8e82-9c1310d22c01/discovery/0.log"
Jul 25 01:35:56.214094 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:56.214055    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-np2wj_7d9483e9-b41a-477b-8e82-9c1310d22c01/discovery/0.log"
Jul 25 01:35:59.903663 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:59.903632    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5c6r_6e35543f-8b3c-4e8c-80fc-9e1a47246e85/kube-multus-additional-cni-plugins/0.log"
Jul 25 01:35:59.922988 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:59.922957    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5c6r_6e35543f-8b3c-4e8c-80fc-9e1a47246e85/egress-router-binary-copy/0.log"
Jul 25 01:35:59.942353 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:59.942332    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5c6r_6e35543f-8b3c-4e8c-80fc-9e1a47246e85/cni-plugins/0.log"
Jul 25 01:35:59.964580 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:59.964559    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5c6r_6e35543f-8b3c-4e8c-80fc-9e1a47246e85/bond-cni-plugin/0.log"
Jul 25 01:35:59.983896 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:35:59.983878    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5c6r_6e35543f-8b3c-4e8c-80fc-9e1a47246e85/routeoverride-cni/0.log"
Jul 25 01:36:00.005473 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:36:00.005452    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5c6r_6e35543f-8b3c-4e8c-80fc-9e1a47246e85/whereabouts-cni-bincopy/0.log"
Jul 25 01:36:00.025297 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:36:00.025279    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b5c6r_6e35543f-8b3c-4e8c-80fc-9e1a47246e85/whereabouts-cni/0.log"
Jul 25 01:36:00.399827 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:36:00.399799    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-swm5k_dc2269d8-bbb4-49f6-8f13-823bd39398f5/kube-multus/0.log"
Jul 25 01:36:00.460243 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:36:00.460217    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bwhm2_f4c7b19e-0f50-493b-a39c-be861ced0055/network-metrics-daemon/0.log"
Jul 25 01:36:00.478633 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:36:00.478614    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bwhm2_f4c7b19e-0f50-493b-a39c-be861ced0055/kube-rbac-proxy/0.log"
Jul 25 01:36:01.355827 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:36:01.355804    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56m27_027dd448-88fa-465e-b7e1-90d9db5cd825/ovn-controller/0.log"
Jul 25 01:36:01.371748 ip-10-0-140-134 kubenswrapper[2566]: I0725 01:36:01.371730    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56m27_027dd448-88fa-465e-b7e1-90d9db5cd825/ovn-acl-logging/0.log"