Jul 13 18:29:55.507346 ip-10-0-139-206 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 13 18:29:55.507362 ip-10-0-139-206 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 13 18:29:55.507371 ip-10-0-139-206 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 13 18:29:55.507700 ip-10-0-139-206 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 13 18:30:05.517105 ip-10-0-139-206 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 13 18:30:05.517119 ip-10-0-139-206 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 996e6214e8ab43f191c4cb921a6bacba --
Jul 13 18:31:58.762932 ip-10-0-139-206 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 18:31:59.118459 ip-10-0-139-206 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 18:31:59.118459 ip-10-0-139-206 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 18:31:59.118459 ip-10-0-139-206 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 18:31:59.118459 ip-10-0-139-206 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 18:31:59.118459 ip-10-0-139-206 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 18:31:59.120132 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.119995    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 18:31:59.125411 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125395    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:31:59.125411 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125412    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125416    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125420    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125422    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125425    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125428    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125430    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125434    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125436    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125439    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125441    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125444    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125446    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125449    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125456    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125459    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125461    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125464    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125466    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125469    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:31:59.125480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125472    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125474    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125476    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125479    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125482    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125485    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125487    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125490    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125492    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125495    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125497    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125499    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125502    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125504    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125507    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125509    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125511    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125514    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125517    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:31:59.125998 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125519    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125522    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125525    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125527    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125530    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125532    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125534    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125536    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125539    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125541    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125543    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125546    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125548    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125550    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125553    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125556    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125559    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125561    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125563    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125566    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:31:59.126451 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125568    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125571    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125573    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125575    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125578    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125582    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125586    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125588    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125591    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125593    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125597    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125602    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125606    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125609    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125612    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125615    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125617    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125620    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125623    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:31:59.126953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125626    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125628    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125631    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125633    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125636    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125638    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125641    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125644    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125661    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125665    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125667    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125670    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125672    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125675    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125678    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125680    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125683    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125685    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125688    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:31:59.127405 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.125691    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126377    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126384    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126386    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126389    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126391    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126394    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126398    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126400    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126403    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126406    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126408    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126411    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126414    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126416    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126419    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126421    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126423    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126426    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126429    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:31:59.127874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126431    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126435    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126438    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126440    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126443    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126446    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126448    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126451    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126453    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126455    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126458    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126461    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126463    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126466    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126468    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126471    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126473    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126476    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126478    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126481    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:31:59.128373 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126484    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126486    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126488    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126490    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126493    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126496    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126498    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126501    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126503    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126505    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126508    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126510    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126513    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126516    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126518    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126521    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126524    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126526    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126528    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126531    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:31:59.128876 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126533    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126536    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126538    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126541    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126543    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126545    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126548    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126550    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126552    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126555    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126557    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126560    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126563    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126565    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126567    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126570    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126572    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126577    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126581    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126585    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:31:59.129365 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126588    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126591    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126593    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126596    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126598    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126601    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126603    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126606    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126608    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126610    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126613    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126615    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126619    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126622    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126624    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126628    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126631    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126634    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126637    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:31:59.129860 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.126639    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127072    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127078    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127081    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127084    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127089    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127093    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127095    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127098    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127101    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127105    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127107    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127110    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127112    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127115    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127118    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127120    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127122    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127125    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:31:59.130316 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127127    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127129    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127132    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127134    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127137    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127139    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127141    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127144    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127146    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127148    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127150    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127153    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127155    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127158    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127162    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127165    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127168    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127171    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127173    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127176    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:31:59.130819 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127178    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127181    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127183    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127186    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127190    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127192    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127194    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127197    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127200    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127202    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127204    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127207    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127209    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127212    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127215    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127217    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127220    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127222    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127225    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127227    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:31:59.131309 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127230    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127232    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127235    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127237    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127240    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127242    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127244    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127247    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127249    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127251    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127254    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127256    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127259    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127262    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127264    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127267    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127269    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127272    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127274    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127278    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:31:59.131817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127280    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127282    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127284    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127287    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127289    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127291    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127294    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127296    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127299    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127303    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127306    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127308    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127313    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127316    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127320    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127323    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127326    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127328    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127331    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:31:59.132322 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127333    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.127335    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127399    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127406    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127419    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127435    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127439    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127442    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127447    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127451    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127454    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127457    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127460    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127463    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127466    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127469    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127472    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127475    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127477    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127480    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127484    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127487    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127489    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127492    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 18:31:59.132792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127496    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127501    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127504    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127507    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127510    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127513    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127516    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127519    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127522    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127525    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127529    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127531    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127534    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127537    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127540    2569 flags.go:64] FLAG: --enable-server="true"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127543    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127549    2569 flags.go:64] FLAG: --event-burst="100"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127551    2569 flags.go:64] FLAG: --event-qps="50"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127554    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127557    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127560    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127563    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127566    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127569    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127572    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 13 18:31:59.133359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127575    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127578    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127580    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127583    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127586    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127588    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127591    2569 flags.go:64] FLAG: --feature-gates=""
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127594    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127598    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127601    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127604    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127607    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127610    2569 flags.go:64] FLAG: --help="false"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127613    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127616    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127619    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127621    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127624    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127628    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127631    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127633    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127636    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127639    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 18:31:59.133971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127644    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127660    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127663    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127665    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127668    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127671    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127674    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127677    2569 flags.go:64] FLAG: --lock-file=""
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127680    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127682    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127685    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127691    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127694    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127696    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127699    2569 flags.go:64] FLAG: --logging-format="text"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127702    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127705    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127710    2569 flags.go:64] FLAG: --manifest-url=""
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127712    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127716    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127719    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127727    2569 flags.go:64] FLAG: --max-pods="110"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127730    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127733    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127736    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 18:31:59.134553 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127739    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127741    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127744    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127747    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127753    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127756    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127759    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127763    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127766    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127772    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127775    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127778    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127781    2569 flags.go:64] FLAG: --port="10250"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127784    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127786    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-016c1e219be734897"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127790    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127792    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127795    2569 flags.go:64] FLAG: --register-node="true"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127798    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127801    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127804    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127807    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127809    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127812    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127816    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 18:31:59.135175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127819    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127822    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127825    2569 flags.go:64] FLAG: --runonce="false"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127827    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127830    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127833    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127835    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127838    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127841    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127844    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127847    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127850    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127853    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127856    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127858    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127862    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127866    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127868    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127874    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127876    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127879    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127883    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127886    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127888    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127891    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 18:31:59.135827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127894    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127897    2569 flags.go:64] FLAG: --v="2"
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127901    2569 flags.go:64] FLAG: --version="false"
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127904    2569 flags.go:64] FLAG: --vmodule=""
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127908    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.127911    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128023    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128026    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128029    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128032    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128034    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128036    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128039    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128041    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128044    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128046    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128049    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128052    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128055    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128059    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:31:59.136687 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128062    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128064    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128068    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128071    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128074    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128076    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128079    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128081    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128083    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128086    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128088    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128091    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128093    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128096    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128098    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128101    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128103    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128105    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128109    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128112    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:31:59.137385 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128115    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128118    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128121    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128124    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128126    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128129    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128132    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128134    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128136    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128139    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128141    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128143    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128146    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128148    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128152    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128154    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128157    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128159    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128162    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128164    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:31:59.137898 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128167    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128169    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128172    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128174    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128176    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128179    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128181    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128184    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128186    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128188    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128192    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128194    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128197    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128200    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128203    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128207    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128210    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128212    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128215    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:31:59.138400 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128217    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128219    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128221    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128224    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128226    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128229    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128231    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128236    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128238    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128241    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128243    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128246    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128248    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128251    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128253    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128256    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128258    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128260    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128262    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128265    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:31:59.138975 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128267    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128270    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128272    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128275    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128278    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.128280    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.128997    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.135626    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.135640    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.135678    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.135684    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135743    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135749    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135752    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135755    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:31:59.139461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135758    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135760    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135764    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135767    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135769    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135772    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135774    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135777    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135780    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135782    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135785    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135787    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135790    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135792    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135795    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135797    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135800    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135802    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135805    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135807    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:31:59.139867 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135810    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135813    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135815    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135819    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135823    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135826    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135829    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135832    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135835    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135838    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135840    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135843    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135845    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135848    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135851    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135853    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135855    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135858    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135860    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:31:59.140346 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135863    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135865    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135868    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135871    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135873    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135876    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135878    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135881    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135883    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135886    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135888    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135890    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135893    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135895    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135898    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135900    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135903    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135906    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135908    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135911    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:31:59.140890 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135913    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135915    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135918    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135920    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135922    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135925    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135927    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135929    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135932    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135934    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135936    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135939    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135941    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135944    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135948    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.135989    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136053    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136058    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136063    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136067    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:31:59.141390 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136071    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136075    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136079    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136083    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136088    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136095    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136102    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136108    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136178    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136201    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136207    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136211    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136214    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136217    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136220    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:31:59.141899 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136223    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.136229    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136482    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136487    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136490    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136496    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136499    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136502    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136505    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136507    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136510    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136513    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136515    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136518    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 18:31:59.142297 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136521    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136524    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136526    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136529    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136534    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136537    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136540    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136542    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136545    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136548    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136551    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136553    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136557    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136560    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136563    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136566    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136571    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136574    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136578    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136582    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 18:31:59.142661 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136585    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136588    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136592    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136596    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136601    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136606    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136610    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136614    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136624    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136629    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136633    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136638    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136642    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136646    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136668    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136672    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136676    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136681    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136686    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136691    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 18:31:59.143150 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136695    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136709    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136717    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136722    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136727    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136734    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136741    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136745    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136752    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136757    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136762    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136770    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136774    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136784    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136789    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136793    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136799    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136803    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136808    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136812    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 18:31:59.143640 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136817    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136821    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136827    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136833    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136838    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136842    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136853    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136859    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136864    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136868    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136873    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136878    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136883    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136887    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136891    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136896    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136900    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136907    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136916    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 18:31:59.144144 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136922    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136926    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136931    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136935    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136939    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136943    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136948    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:31:59.136952    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.136961    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.137410    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.139628    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.141016    2569 server.go:1019] "Starting client certificate rotation"
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.141115    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 18:31:59.144636 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.141147    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 18:31:59.145049 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.143417    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 18:31:59.146082 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.146069    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 18:31:59.155496 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.155476    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 13 18:31:59.157609 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.157593    2569 log.go:25] "Validated CRI v1 image API"
Jul 13 18:31:59.158478 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.158462    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 18:31:59.163496 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.163472    2569 fs.go:135] Filesystem UUIDs: map[175ca23b-61f9-4489-8079-124cc3464728:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e2a4f8e7-a316-493c-adac-c97011f9c454:/dev/nvme0n1p3]
Jul 13 18:31:59.163570 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.163494    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 18:31:59.168549 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.168443    2569 manager.go:216] Machine: {Timestamp:2026-07-13 18:31:59.167989405 +0000 UTC m=+0.299277856 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099708 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d50ec1e25499a6ff2a22b50cab662 SystemUUID:ec2d50ec-1e25-499a-6ff2-a22b50cab662 BootID:996e6214-e8ab-43f1-91c4-cb921a6bacba Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e4:79:73:79:03 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e4:79:73:79:03 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:8c:81:ba:df:df Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 18:31:59.168549 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.168541    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 18:31:59.168688 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.168612    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 18:31:59.169454 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.169436    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 18:31:59.171637 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.171613    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 18:31:59.171816 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.171641    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-206.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 18:31:59.173174 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.173164    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 18:31:59.173208 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.173177    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 18:31:59.173208 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.173184    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 18:31:59.173826 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.173816    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 18:31:59.175258 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.175248    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 18:31:59.179416 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.179403    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 18:31:59.179537 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.179528    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 18:31:59.179571 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.179558    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 18:31:59.179601 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.179576    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 18:31:59.179631 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.179612    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 18:31:59.179631 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.179622    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 13 18:31:59.179631 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.179630    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 18:31:59.180156 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.180134    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 18:31:59.180226 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.180171    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 18:31:59.187609 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.187589    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 18:31:59.189560 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.189547    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 18:31:59.189612 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.189578    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 18:31:59.189760 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.189750    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 18:31:59.189798 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.189766    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 18:31:59.189798 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.189772    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 18:31:59.189798 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.189777    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 18:31:59.189798 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.189785    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 18:31:59.189798 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.189790    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 18:31:59.190205 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.190196    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 18:31:59.190205 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.190205    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 18:31:59.190277 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.190211    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 18:31:59.190277 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.190217    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 18:31:59.190277 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.190225    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 18:31:59.190277 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.190232    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 18:31:59.190277 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.190256    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 18:31:59.190277 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.190261    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 18:31:59.192563 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.192541    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-llfb4"
Jul 13 18:31:59.193565 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.193550    2569 server.go:1268] "Started kubelet"
Jul 13 18:31:59.193692 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.193626    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 18:31:59.193692 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.193635    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 18:31:59.193797 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.193720    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 18:31:59.194357 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.194253    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-206.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 18:31:59.194357 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.194261    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-206.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 18:31:59.194357 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.194325    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 18:31:59.194463 ip-10-0-139-206 systemd[1]: Started Kubernetes Kubelet.
Jul 13 18:31:59.195069 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.195059    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 18:31:59.196224 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.196208    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 18:31:59.199327 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.199301    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 18:31:59.199723 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.199708    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-llfb4"
Jul 13 18:31:59.200181 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.200168    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 18:31:59.200230 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.200173    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 18:31:59.200300 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.200283    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:31:59.200508 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.199626    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-206.ec2.internal.18c1ed01c05d6bd9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-206.ec2.internal,UID:ip-10-0-139-206.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-206.ec2.internal,},FirstTimestamp:2026-07-13 18:31:59.193525209 +0000 UTC m=+0.324813660,LastTimestamp:2026-07-13 18:31:59.193525209 +0000 UTC m=+0.324813660,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-206.ec2.internal,}"
Jul 13 18:31:59.200981 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.200965    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 18:31:59.201062 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.200985    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 18:31:59.201062 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.200965    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 18:31:59.201062 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.201058    2569 factory.go:55] Registering systemd factory
Jul 13 18:31:59.201219 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.201073    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 13 18:31:59.201219 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.201163    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 18:31:59.201219 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.201174    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 18:31:59.201556 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.201538    2569 factory.go:153] Registering CRI-O factory
Jul 13 18:31:59.201640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.201559    2569 factory.go:223] Registration of the crio container factory successfully
Jul 13 18:31:59.201640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.201623    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 18:31:59.201745 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.201645    2569 factory.go:103] Registering Raw factory
Jul 13 18:31:59.201745 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.201682    2569 manager.go:1196] Started watching for new ooms in manager
Jul 13 18:31:59.202469 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.202448    2569 manager.go:349] Starting recovery of all containers
Jul 13 18:31:59.208040 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.208020    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:31:59.212313 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.212294    2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-206.ec2.internal\" not found" node="ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.213677 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.213510    2569 manager.go:354] Recovery completed
Jul 13 18:31:59.218359 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.218344    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:31:59.221309 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.221294    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:31:59.221355 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.221329    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:31:59.221355 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.221346    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:31:59.221907 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.221896    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 18:31:59.221907 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.221903    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:31:59.221992 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.221907    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 18:31:59.221992 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.221923    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 18:31:59.222619 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.222606    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:31:59.222705 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.222632    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:31:59.222705 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.222642    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:31:59.223948 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.223936    2569 policy_none.go:49] "None policy: Start"
Jul 13 18:31:59.223990 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.223952    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 18:31:59.223990 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.223963    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 18:31:59.225950 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.225940    2569 policy_none.go:47] "Start"
Jul 13 18:31:59.263212 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.263194    2569 manager.go:337] "Starting Device Plugin manager"
Jul 13 18:31:59.263307 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.263233    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 18:31:59.263307 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.263246    2569 server.go:87] "Starting device plugin registration server"
Jul 13 18:31:59.263474 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.263463    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 18:31:59.263506 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.263476    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 18:31:59.263613 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.263595    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 18:31:59.263700 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.263687    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 18:31:59.263700 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.263700    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 18:31:59.264167 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.264149    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 18:31:59.264260 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.264190    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:31:59.342495 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.342464    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 18:31:59.344627 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.343805    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 18:31:59.344627 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.343833    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 18:31:59.344627 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.343862    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 18:31:59.344627 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.343898    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 18:31:59.346560 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.346543    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:31:59.364469 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.364446    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:31:59.365869 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.365855    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:31:59.365944 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.365886    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:31:59.365944 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.365899    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:31:59.365944 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.365928    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.375389 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.375345    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.375389 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.375368    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-206.ec2.internal\": node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:31:59.387843 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.387827    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:31:59.444722 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.444696    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal"]
Jul 13 18:31:59.444797 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.444783    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:31:59.445738 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.445721    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:31:59.445806 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.445752    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:31:59.445806 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.445763    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:31:59.448073 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.448060    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:31:59.448190 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.448174    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.448230 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.448210    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:31:59.448787 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.448772    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:31:59.448861 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.448797    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:31:59.448861 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.448807    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:31:59.448861 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.448819    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:31:59.448861 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.448837    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:31:59.448861 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.448846    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:31:59.451035 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.451019    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.451125 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.451042    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 18:31:59.451709 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.451694    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 18:31:59.451773 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.451722    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 18:31:59.451773 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.451732    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID"
Jul 13 18:31:59.477243 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.477218    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-206.ec2.internal\" not found" node="ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.481664 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.481633    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-206.ec2.internal\" not found" node="ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.488689 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.488674    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:31:59.502730 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.502710    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/588f587a0ae505f4fc590f0ee32209fe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"588f587a0ae505f4fc590f0ee32209fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.502778 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.502734    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48897f40f8f0e850d106dd2f49700b6a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-206.ec2.internal\" (UID: \"48897f40f8f0e850d106dd2f49700b6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.502778 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.502760    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/588f587a0ae505f4fc590f0ee32209fe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"588f587a0ae505f4fc590f0ee32209fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.589643 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.589614    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:31:59.602983 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.602961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/588f587a0ae505f4fc590f0ee32209fe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"588f587a0ae505f4fc590f0ee32209fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.603061 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.602988    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/588f587a0ae505f4fc590f0ee32209fe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"588f587a0ae505f4fc590f0ee32209fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.603061 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.603003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48897f40f8f0e850d106dd2f49700b6a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-206.ec2.internal\" (UID: \"48897f40f8f0e850d106dd2f49700b6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.603061 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.603039    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48897f40f8f0e850d106dd2f49700b6a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-206.ec2.internal\" (UID: \"48897f40f8f0e850d106dd2f49700b6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.603061 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.603045    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/588f587a0ae505f4fc590f0ee32209fe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"588f587a0ae505f4fc590f0ee32209fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.603203 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.603088    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/588f587a0ae505f4fc590f0ee32209fe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"588f587a0ae505f4fc590f0ee32209fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.689932 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.689849    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:31:59.779361 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.779325    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.785376 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:31:59.784639    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal"
Jul 13 18:31:59.790775 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.790739    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:31:59.891312 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.891281    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:31:59.992612 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:31:59.992545    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:32:00.097306 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:00.097256    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:32:00.142113 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.142080    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 18:32:00.142584 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.142250    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 18:32:00.142584 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.142250    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 18:32:00.197914 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:00.197876    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:32:00.205869 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.205840    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 18:26:59 +0000 UTC" deadline="2027-12-11 01:23:58.348160704 +0000 UTC"
Jul 13 18:32:00.205962 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.205869    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12366h51m58.142296212s"
Jul 13 18:32:00.300193 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:00.300119    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:32:00.403621 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:00.403584    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:32:00.507541 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:00.507491    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:32:00.519789 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.519760    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:32:00.607868 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:00.607784    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:32:00.708624 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:00.708594    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found"
Jul 13 18:32:00.734374 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.734346    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:32:00.801270 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.801239    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal"
Jul 13 18:32:00.814186 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.814164    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 18:32:00.815097 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.815084    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal"
Jul 13 18:32:00.826507 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.826490    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 18:32:00.993360 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:00.993332    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:32:01.004668 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:01.004624    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48897f40f8f0e850d106dd2f49700b6a.slice/crio-a7bb2bb5b53fdbaff824525073607c00bb3d3c431be0b58aee9c1a70ced82d63 WatchSource:0}: Error finding container a7bb2bb5b53fdbaff824525073607c00bb3d3c431be0b58aee9c1a70ced82d63: Status 404 returned error can't find the container with id a7bb2bb5b53fdbaff824525073607c00bb3d3c431be0b58aee9c1a70ced82d63
Jul 13 18:32:01.009187 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.009171    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 18:32:01.181974 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.181879    2569 apiserver.go:52] "Watching apiserver"
Jul 13 18:32:01.191134 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.190436    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 18:32:01.191747 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.191710    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-zvvkh","openshift-network-diagnostics/network-check-target-blbvt","openshift-network-operator/iptables-alerter-t85bl","kube-system/konnectivity-agent-9fx5s","openshift-dns/node-resolver-88fdd","openshift-image-registry/node-ca-qs9wn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal","openshift-multus/multus-c5db7","openshift-ovn-kubernetes/ovnkube-node-r6wmn","kube-system/global-pull-secret-syncer-ffsc6","kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh","openshift-cluster-node-tuning-operator/tuned-m6256","openshift-multus/multus-additional-cni-plugins-fblxn"]
Jul 13 18:32:01.195767 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.195749    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:01.195851 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.195831    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:01.197999 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.197981    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.200288 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.200270    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.200773 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.200758    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 18:32:01.200838 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.200822    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9pvbg\""
Jul 13 18:32:01.200885 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.200822    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 18:32:01.201525 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.201512    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 18:32:01.202539 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.202520    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:01.203961 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.203942    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 18:32:01.204062 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.203969    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 18:32:01.204062 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.203942    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 18:32:01.204173 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.204057    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rcxzj\""
Jul 13 18:32:01.204889 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.204866    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.205963 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.205804    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 18:32:01.205963 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.205836    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fb4z4\""
Jul 13 18:32:01.205963 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.205843    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 18:32:01.207227 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.207207    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.208155 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.208138    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 18:32:01.208239 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.208165    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 18:32:01.208287 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.208259    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gcfsk\""
Jul 13 18:32:01.209871 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.209855    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 18:32:01.209948 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.209855    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.210386 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.210371    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 18:32:01.212224 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.212207    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.213087 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213066    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 18:32:01.213169 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213087    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qdhnz\""
Jul 13 18:32:01.213392 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213376    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 18:32:01.213478 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213462    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 18:32:01.213546 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213467    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 18:32:01.213601 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213556    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 18:32:01.213714 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213698    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-tuned\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.213792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213726    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e27fd358-8fdd-4706-b5b1-f234e710484a-tmp\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.213792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213746    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-lib-modules\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.213901 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213791    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsjnn\" (UniqueName: \"kubernetes.io/projected/e27fd358-8fdd-4706-b5b1-f234e710484a-kube-api-access-nsjnn\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.213901 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213827    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d10fd769-b61e-4f9f-9a82-25ff768d4533-iptables-alerter-script\") pod \"iptables-alerter-t85bl\" (UID: \"d10fd769-b61e-4f9f-9a82-25ff768d4533\") " pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.213901 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213854    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/edb49c7e-ea32-4018-9f70-6d25d8b680c4-agent-certs\") pod \"konnectivity-agent-9fx5s\" (UID: \"edb49c7e-ea32-4018-9f70-6d25d8b680c4\") " pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:01.213901 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213878    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fe11421-9613-4ca6-8c39-c44b3682f360-kubelet-config\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:01.214100 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213905    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fe11421-9613-4ca6-8c39-c44b3682f360-dbus\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:01.214100 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213938    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-kubernetes\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.214100 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-systemd\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.214100 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.213988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz6v9\" (UniqueName: \"kubernetes.io/projected/d10fd769-b61e-4f9f-9a82-25ff768d4533-kube-api-access-cz6v9\") pod \"iptables-alerter-t85bl\" (UID: \"d10fd769-b61e-4f9f-9a82-25ff768d4533\") " pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.214100 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214014    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/edb49c7e-ea32-4018-9f70-6d25d8b680c4-konnectivity-ca\") pod \"konnectivity-agent-9fx5s\" (UID: \"edb49c7e-ea32-4018-9f70-6d25d8b680c4\") " pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:01.214100 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214040    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:01.214100 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-sysctl-d\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.214100 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214089    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-run\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.214431 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214112    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-sys\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.214431 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214137    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-host\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.214431 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214160    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d10fd769-b61e-4f9f-9a82-25ff768d4533-host-slash\") pod \"iptables-alerter-t85bl\" (UID: \"d10fd769-b61e-4f9f-9a82-25ff768d4533\") " pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.214431 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214185    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-modprobe-d\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.214431 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-sysconfig\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.214431 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214275    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-sysctl-conf\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.214431 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214301    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-var-lib-kubelet\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.214720 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214640    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:01.214720 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214663    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-scm8l\""
Jul 13 18:32:01.214720 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.214676    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 18:32:01.214894 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.214761    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:01.215029 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.215012    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 18:32:01.215106 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.215017    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 18:32:01.215106 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.215046    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 18:32:01.215453 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.215419    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 18:32:01.215453 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.215425    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 18:32:01.215675 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.215500    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cw29c\""
Jul 13 18:32:01.215675 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.215566    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 18:32:01.217262 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.217243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:01.217362 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.217299    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:01.218312 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.218294    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 18:32:01.219829 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.219815    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.222246 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.222225    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mkd2t\""
Jul 13 18:32:01.222454 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.222434    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.222537 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.222475    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 18:32:01.225399 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.225346    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 18:32:01.225881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.225862    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 18:32:01.226189 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.225992    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sb9x8\""
Jul 13 18:32:01.226189 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.226000    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 18:32:01.248450 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.248427    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5m2xf"
Jul 13 18:32:01.261514 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.261494    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5m2xf"
Jul 13 18:32:01.302784 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.302764    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 18:32:01.306389 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.306373    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 18:32:01.314822 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.314802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-var-lib-cni-bin\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.314915 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.314838    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-sysconfig\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.314915 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.314860    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.314915 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.314876    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-socket-dir-parent\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.314915 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.314892    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw7np\" (UniqueName: \"kubernetes.io/projected/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-kube-api-access-tw7np\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.315114 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.314912    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-kubelet\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.315114 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.314932    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:01.315114 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.314942    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-sysconfig\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.315114 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.314948    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-run-multus-certs\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.315114 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315014    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fe11421-9613-4ca6-8c39-c44b3682f360-dbus\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:01.315114 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315048    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsjnn\" (UniqueName: \"kubernetes.io/projected/e27fd358-8fdd-4706-b5b1-f234e710484a-kube-api-access-nsjnn\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.315114 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315072    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8d9z\" (UniqueName: \"kubernetes.io/projected/91a5e102-1948-414f-bfc4-a468dfe9f3ba-kube-api-access-l8d9z\") pod \"node-resolver-88fdd\" (UID: \"91a5e102-1948-414f-bfc4-a468dfe9f3ba\") " pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.315314 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315111    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-cni-bin\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.315314 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bchnm\" (UniqueName: \"kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm\") pod \"network-check-target-blbvt\" (UID: \"99b3b7e2-f203-4325-945a-214b3a386b4a\") " pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:01.315314 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d153a23e-6499-4764-ad85-666ef8b2dc72-serviceca\") pod \"node-ca-qs9wn\" (UID: \"d153a23e-6499-4764-ad85-666ef8b2dc72\") " pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.315314 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315218    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fe11421-9613-4ca6-8c39-c44b3682f360-dbus\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:01.315314 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315231    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d9j2\" (UniqueName: \"kubernetes.io/projected/bf25eded-5569-4d4d-be04-4eb93300fcf2-kube-api-access-4d9j2\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.315314 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315263    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-sysctl-d\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315312    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-ovnkube-script-lib\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-cnibin\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315350    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-os-release\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315371    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-modprobe-d\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315386    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-sysctl-conf\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315402    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-systemd-units\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315411    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-sysctl-d\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-var-lib-openvswitch\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-modprobe-d\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315457    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-run-openvswitch\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315485    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-log-socket\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.315527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315520    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d153a23e-6499-4764-ad85-666ef8b2dc72-host\") pod \"node-ca-qs9wn\" (UID: \"d153a23e-6499-4764-ad85-666ef8b2dc72\") " pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315541    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf25eded-5569-4d4d-be04-4eb93300fcf2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315529    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-sysctl-conf\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315557    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a5e102-1948-414f-bfc4-a468dfe9f3ba-tmp-dir\") pod \"node-resolver-88fdd\" (UID: \"91a5e102-1948-414f-bfc4-a468dfe9f3ba\") " pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315575    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-var-lib-cni-multus\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315596    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315645    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/edb49c7e-ea32-4018-9f70-6d25d8b680c4-agent-certs\") pod \"konnectivity-agent-9fx5s\" (UID: \"edb49c7e-ea32-4018-9f70-6d25d8b680c4\") " pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315699    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-systemd\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-run\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315827    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-run\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315829    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-systemd\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-sys\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315865    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-run-netns\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315872    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-sys\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315883    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315900    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-cni-netd\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315915    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf25eded-5569-4d4d-be04-4eb93300fcf2-cni-binary-copy\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.316079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-env-overrides\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315968    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bdq5\" (UniqueName: \"kubernetes.io/projected/698c185e-1b6a-43a7-97fb-725ad56c0962-kube-api-access-8bdq5\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.315998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-run-netns\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316022    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-socket-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316038    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-device-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316056    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-etc-openvswitch\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316107    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-os-release\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316138    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-var-lib-kubelet\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316164    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd5f7\" (UniqueName: \"kubernetes.io/projected/108324b4-322a-4ba1-9b6a-16e3ba7fea94-kube-api-access-fd5f7\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-ovnkube-config\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-sys-fs\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316222    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-cnibin\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316248    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d10fd769-b61e-4f9f-9a82-25ff768d4533-iptables-alerter-script\") pod \"iptables-alerter-t85bl\" (UID: \"d10fd769-b61e-4f9f-9a82-25ff768d4533\") " pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-kubernetes\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316283    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91a5e102-1948-414f-bfc4-a468dfe9f3ba-hosts-file\") pod \"node-resolver-88fdd\" (UID: \"91a5e102-1948-414f-bfc4-a468dfe9f3ba\") " pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316305    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp5sf\" (UniqueName: \"kubernetes.io/projected/d153a23e-6499-4764-ad85-666ef8b2dc72-kube-api-access-cp5sf\") pod \"node-ca-qs9wn\" (UID: \"d153a23e-6499-4764-ad85-666ef8b2dc72\") " pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.316935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316331    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-run-k8s-cni-cncf-io\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316352    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-kubernetes\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316354    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-etc-kubernetes\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316404    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-system-cni-dir\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/edb49c7e-ea32-4018-9f70-6d25d8b680c4-konnectivity-ca\") pod \"konnectivity-agent-9fx5s\" (UID: \"edb49c7e-ea32-4018-9f70-6d25d8b680c4\") " pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316537    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-host\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-host\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316612    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-tuned\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316643    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-ovn-node-metrics-cert\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316683    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/108324b4-322a-4ba1-9b6a-16e3ba7fea94-cni-binary-copy\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-conf-dir\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316736    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d10fd769-b61e-4f9f-9a82-25ff768d4533-host-slash\") pod \"iptables-alerter-t85bl\" (UID: \"d10fd769-b61e-4f9f-9a82-25ff768d4533\") " pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316762    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-var-lib-kubelet\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316785    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-slash\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316866    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-run-ovn\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.316936    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d10fd769-b61e-4f9f-9a82-25ff768d4533-iptables-alerter-script\") pod \"iptables-alerter-t85bl\" (UID: \"d10fd769-b61e-4f9f-9a82-25ff768d4533\") " pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317015    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-node-log\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.317635 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-system-cni-dir\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317061    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-cni-dir\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317053    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d10fd769-b61e-4f9f-9a82-25ff768d4533-host-slash\") pod \"iptables-alerter-t85bl\" (UID: \"d10fd769-b61e-4f9f-9a82-25ff768d4533\") " pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317085    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-etc-selinux\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-lib-modules\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317124    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-var-lib-kubelet\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317072    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/edb49c7e-ea32-4018-9f70-6d25d8b680c4-konnectivity-ca\") pod \"konnectivity-agent-9fx5s\" (UID: \"edb49c7e-ea32-4018-9f70-6d25d8b680c4\") " pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317130    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqxmz\" (UniqueName: \"kubernetes.io/projected/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-kube-api-access-xqxmz\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317256    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-daemon-config\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317311    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e27fd358-8fdd-4706-b5b1-f234e710484a-lib-modules\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fe11421-9613-4ca6-8c39-c44b3682f360-kubelet-config\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317521    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317553    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bf25eded-5569-4d4d-be04-4eb93300fcf2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317579    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-hostroot\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317595    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fe11421-9613-4ca6-8c39-c44b3682f360-kubelet-config\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-registration-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.318280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317637    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz6v9\" (UniqueName: \"kubernetes.io/projected/d10fd769-b61e-4f9f-9a82-25ff768d4533-kube-api-access-cz6v9\") pod \"iptables-alerter-t85bl\" (UID: \"d10fd769-b61e-4f9f-9a82-25ff768d4533\") " pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.318846 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:01.318846 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.317789    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:01.318846 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.317853    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret podName:8fe11421-9613-4ca6-8c39-c44b3682f360 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:01.81783025 +0000 UTC m=+2.949118690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret") pod "global-pull-secret-syncer-ffsc6" (UID: "8fe11421-9613-4ca6-8c39-c44b3682f360") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:01.318846 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317790    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e27fd358-8fdd-4706-b5b1-f234e710484a-tmp\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.318846 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.317898    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-run-systemd\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.319331 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.319307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e27fd358-8fdd-4706-b5b1-f234e710484a-etc-tuned\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.319526 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.319507    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/edb49c7e-ea32-4018-9f70-6d25d8b680c4-agent-certs\") pod \"konnectivity-agent-9fx5s\" (UID: \"edb49c7e-ea32-4018-9f70-6d25d8b680c4\") " pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:01.319933 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.319917    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e27fd358-8fdd-4706-b5b1-f234e710484a-tmp\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.325122 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.325105    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsjnn\" (UniqueName: \"kubernetes.io/projected/e27fd358-8fdd-4706-b5b1-f234e710484a-kube-api-access-nsjnn\") pod \"tuned-m6256\" (UID: \"e27fd358-8fdd-4706-b5b1-f234e710484a\") " pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.329907 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.329891    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz6v9\" (UniqueName: \"kubernetes.io/projected/d10fd769-b61e-4f9f-9a82-25ff768d4533-kube-api-access-cz6v9\") pod \"iptables-alerter-t85bl\" (UID: \"d10fd769-b61e-4f9f-9a82-25ff768d4533\") " pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.349506 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.349422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" event={"ID":"48897f40f8f0e850d106dd2f49700b6a","Type":"ContainerStarted","Data":"a7bb2bb5b53fdbaff824525073607c00bb3d3c431be0b58aee9c1a70ced82d63"}
Jul 13 18:32:01.418222 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418187    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-ovnkube-config\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.418222 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418222    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-sys-fs\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.418458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418238    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-cnibin\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.418458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91a5e102-1948-414f-bfc4-a468dfe9f3ba-hosts-file\") pod \"node-resolver-88fdd\" (UID: \"91a5e102-1948-414f-bfc4-a468dfe9f3ba\") " pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.418458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418274    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp5sf\" (UniqueName: \"kubernetes.io/projected/d153a23e-6499-4764-ad85-666ef8b2dc72-kube-api-access-cp5sf\") pod \"node-ca-qs9wn\" (UID: \"d153a23e-6499-4764-ad85-666ef8b2dc72\") " pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.418458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418354    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91a5e102-1948-414f-bfc4-a468dfe9f3ba-hosts-file\") pod \"node-resolver-88fdd\" (UID: \"91a5e102-1948-414f-bfc4-a468dfe9f3ba\") " pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.418458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418405    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-run-k8s-cni-cncf-io\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.418458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418440    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-etc-kubernetes\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418471    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-system-cni-dir\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418501    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-sys-fs\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418501    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-ovn-node-metrics-cert\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418552    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-run-k8s-cni-cncf-io\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/108324b4-322a-4ba1-9b6a-16e3ba7fea94-cni-binary-copy\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-conf-dir\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418640    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-system-cni-dir\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418657    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-slash\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418684    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-run-ovn\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418723    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-node-log\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418748    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-system-cni-dir\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.418774 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418773    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-cni-dir\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418798    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-etc-selinux\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418825    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqxmz\" (UniqueName: \"kubernetes.io/projected/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-kube-api-access-xqxmz\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-daemon-config\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418861    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-ovnkube-config\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418881    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418930    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-node-log\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418942    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418954    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bf25eded-5569-4d4d-be04-4eb93300fcf2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419001    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-system-cni-dir\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419032    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-hostroot\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419052    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-registration-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419080    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-run-systemd\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419101    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-var-lib-cni-bin\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419129    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419171    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-socket-dir-parent\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419217    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw7np\" (UniqueName: \"kubernetes.io/projected/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-kube-api-access-tw7np\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.419367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419236    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-kubelet\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-run-multus-certs\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419304    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8d9z\" (UniqueName: \"kubernetes.io/projected/91a5e102-1948-414f-bfc4-a468dfe9f3ba-kube-api-access-l8d9z\") pod \"node-resolver-88fdd\" (UID: \"91a5e102-1948-414f-bfc4-a468dfe9f3ba\") " pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419330    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-cni-bin\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419355    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-cnibin\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419371    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchnm\" (UniqueName: \"kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm\") pod \"network-check-target-blbvt\" (UID: \"99b3b7e2-f203-4325-945a-214b3a386b4a\") " pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419403    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d153a23e-6499-4764-ad85-666ef8b2dc72-serviceca\") pod \"node-ca-qs9wn\" (UID: \"d153a23e-6499-4764-ad85-666ef8b2dc72\") " pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419420    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-var-lib-cni-bin\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419420    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bf25eded-5569-4d4d-be04-4eb93300fcf2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419432    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d9j2\" (UniqueName: \"kubernetes.io/projected/bf25eded-5569-4d4d-be04-4eb93300fcf2-kube-api-access-4d9j2\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419459    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-ovnkube-script-lib\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419576    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-socket-dir-parent\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.419608    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419626    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-cni-bin\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.419712    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs podName:698c185e-1b6a-43a7-97fb-725ad56c0962 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:01.919694002 +0000 UTC m=+3.050982443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs") pod "network-metrics-daemon-zvvkh" (UID: "698c185e-1b6a-43a7-97fb-725ad56c0962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419725    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-etc-selinux\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.420680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419914    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-run-multus-certs\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419915    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-conf-dir\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419915    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/108324b4-322a-4ba1-9b6a-16e3ba7fea94-cni-binary-copy\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419934    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-slash\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.418601    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-etc-kubernetes\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.419965    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-run-ovn\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420083    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-cnibin\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420090    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-kubelet\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420117    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-os-release\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420126    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-hostroot\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420117    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-run-systemd\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420181    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-registration-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420200    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-cnibin\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-systemd-units\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420241    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d153a23e-6499-4764-ad85-666ef8b2dc72-serviceca\") pod \"node-ca-qs9wn\" (UID: \"d153a23e-6499-4764-ad85-666ef8b2dc72\") " pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420248    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-systemd-units\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420247    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-var-lib-openvswitch\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420283    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-var-lib-openvswitch\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.421521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420289    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-cni-dir\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420287    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-os-release\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420321    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-run-openvswitch\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-run-openvswitch\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-log-socket\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420401    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d153a23e-6499-4764-ad85-666ef8b2dc72-host\") pod \"node-ca-qs9wn\" (UID: \"d153a23e-6499-4764-ad85-666ef8b2dc72\") " pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420405    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-log-socket\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420435    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf25eded-5569-4d4d-be04-4eb93300fcf2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420446    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d153a23e-6499-4764-ad85-666ef8b2dc72-host\") pod \"node-ca-qs9wn\" (UID: \"d153a23e-6499-4764-ad85-666ef8b2dc72\") " pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420449    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a5e102-1948-414f-bfc4-a468dfe9f3ba-tmp-dir\") pod \"node-resolver-88fdd\" (UID: \"91a5e102-1948-414f-bfc4-a468dfe9f3ba\") " pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420560    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-var-lib-cni-multus\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420609    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420640    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-ovnkube-script-lib\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420682    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-var-lib-cni-multus\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420703    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420810    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-run-netns\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420850    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420864    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a5e102-1948-414f-bfc4-a468dfe9f3ba-tmp-dir\") pod \"node-resolver-88fdd\" (UID: \"91a5e102-1948-414f-bfc4-a468dfe9f3ba\") " pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420879    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-run-netns\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420928    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-cni-netd\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420963    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf25eded-5569-4d4d-be04-4eb93300fcf2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420963    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-cni-netd\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.420992    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf25eded-5569-4d4d-be04-4eb93300fcf2-cni-binary-copy\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421008    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-env-overrides\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421060    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bdq5\" (UniqueName: \"kubernetes.io/projected/698c185e-1b6a-43a7-97fb-725ad56c0962-kube-api-access-8bdq5\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421086    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-run-netns\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-run-netns\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-socket-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421189    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-device-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421213    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-etc-openvswitch\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421239    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-os-release\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421264    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-var-lib-kubelet\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421293    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd5f7\" (UniqueName: \"kubernetes.io/projected/108324b4-322a-4ba1-9b6a-16e3ba7fea94-kube-api-access-fd5f7\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.422640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-etc-openvswitch\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.423160 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421313    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-socket-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.423160 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421359    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/108324b4-322a-4ba1-9b6a-16e3ba7fea94-multus-daemon-config\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.423160 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421369    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-device-dir\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.423160 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421383    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-env-overrides\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.423160 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421404    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/108324b4-322a-4ba1-9b6a-16e3ba7fea94-host-var-lib-kubelet\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.423160 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421385    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bf25eded-5569-4d4d-be04-4eb93300fcf2-os-release\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.423160 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421768    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-ovn-node-metrics-cert\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.423160 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.421979    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf25eded-5569-4d4d-be04-4eb93300fcf2-cni-binary-copy\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.431245 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.431221    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:32:01.431356 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.431252    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:32:01.431356 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.431267    2569 projected.go:196] Error preparing data for projected volume kube-api-access-bchnm for pod openshift-network-diagnostics/network-check-target-blbvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:01.431356 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.431340    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm podName:99b3b7e2-f203-4325-945a-214b3a386b4a nodeName:}" failed. No retries permitted until 2026-07-13 18:32:01.931321696 +0000 UTC m=+3.062610155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bchnm" (UniqueName: "kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm") pod "network-check-target-blbvt" (UID: "99b3b7e2-f203-4325-945a-214b3a386b4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:01.433281 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.433228    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp5sf\" (UniqueName: \"kubernetes.io/projected/d153a23e-6499-4764-ad85-666ef8b2dc72-kube-api-access-cp5sf\") pod \"node-ca-qs9wn\" (UID: \"d153a23e-6499-4764-ad85-666ef8b2dc72\") " pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.433484 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.433455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bdq5\" (UniqueName: \"kubernetes.io/projected/698c185e-1b6a-43a7-97fb-725ad56c0962-kube-api-access-8bdq5\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:01.433800 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.433779    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqxmz\" (UniqueName: \"kubernetes.io/projected/10a8e7a5-f7f3-4b20-847a-f8563ff427c7-kube-api-access-xqxmz\") pod \"ovnkube-node-r6wmn\" (UID: \"10a8e7a5-f7f3-4b20-847a-f8563ff427c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.433885 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.433797    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d9j2\" (UniqueName: \"kubernetes.io/projected/bf25eded-5569-4d4d-be04-4eb93300fcf2-kube-api-access-4d9j2\") pod \"multus-additional-cni-plugins-fblxn\" (UID: \"bf25eded-5569-4d4d-be04-4eb93300fcf2\") " pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.433885 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.433805    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw7np\" (UniqueName: \"kubernetes.io/projected/cb44eda8-27b8-4e19-a0e7-6e3ffa934de6-kube-api-access-tw7np\") pod \"aws-ebs-csi-driver-node-7h5xh\" (UID: \"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.434095 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.434069    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd5f7\" (UniqueName: \"kubernetes.io/projected/108324b4-322a-4ba1-9b6a-16e3ba7fea94-kube-api-access-fd5f7\") pod \"multus-c5db7\" (UID: \"108324b4-322a-4ba1-9b6a-16e3ba7fea94\") " pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.434371 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.434351    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8d9z\" (UniqueName: \"kubernetes.io/projected/91a5e102-1948-414f-bfc4-a468dfe9f3ba-kube-api-access-l8d9z\") pod \"node-resolver-88fdd\" (UID: \"91a5e102-1948-414f-bfc4-a468dfe9f3ba\") " pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.507408 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.507372    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m6256"
Jul 13 18:32:01.516282 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.516257    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t85bl"
Jul 13 18:32:01.518107 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:01.518074    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode27fd358_8fdd_4706_b5b1_f234e710484a.slice/crio-03e03e69163fbee30cc29da1f3d3577bf5c6309cc072b591f5bca4e47a599efd WatchSource:0}: Error finding container 03e03e69163fbee30cc29da1f3d3577bf5c6309cc072b591f5bca4e47a599efd: Status 404 returned error can't find the container with id 03e03e69163fbee30cc29da1f3d3577bf5c6309cc072b591f5bca4e47a599efd
Jul 13 18:32:01.526773 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.526753    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:01.529789 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.529766    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-88fdd"
Jul 13 18:32:01.533900 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:01.533864    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10fd769_b61e_4f9f_9a82_25ff768d4533.slice/crio-c4cb7be59dff817493ee81f2765ffc1c2e9fdfc03c067aa624fa3b8e16865b78 WatchSource:0}: Error finding container c4cb7be59dff817493ee81f2765ffc1c2e9fdfc03c067aa624fa3b8e16865b78: Status 404 returned error can't find the container with id c4cb7be59dff817493ee81f2765ffc1c2e9fdfc03c067aa624fa3b8e16865b78
Jul 13 18:32:01.534454 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.534437    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qs9wn"
Jul 13 18:32:01.538873 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:01.538845    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb49c7e_ea32_4018_9f70_6d25d8b680c4.slice/crio-4b14d5cd2469ff2bfef5921315f13256b3a29e767f3db3b39bee11ef1fb94cb4 WatchSource:0}: Error finding container 4b14d5cd2469ff2bfef5921315f13256b3a29e767f3db3b39bee11ef1fb94cb4: Status 404 returned error can't find the container with id 4b14d5cd2469ff2bfef5921315f13256b3a29e767f3db3b39bee11ef1fb94cb4
Jul 13 18:32:01.539842 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:01.539820    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a5e102_1948_414f_bfc4_a468dfe9f3ba.slice/crio-d7ac922ec2d7943339d5a74fc5a103fe51a2fe3314ad17d4e68dcc0249df8a0c WatchSource:0}: Error finding container d7ac922ec2d7943339d5a74fc5a103fe51a2fe3314ad17d4e68dcc0249df8a0c: Status 404 returned error can't find the container with id d7ac922ec2d7943339d5a74fc5a103fe51a2fe3314ad17d4e68dcc0249df8a0c
Jul 13 18:32:01.539842 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.539829    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:01.545461 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:01.545414    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd153a23e_6499_4764_ad85_666ef8b2dc72.slice/crio-84a7ce7aa1831cffd9c6817731037e2b6c8761dcbe21870c8699d246cbf7435d WatchSource:0}: Error finding container 84a7ce7aa1831cffd9c6817731037e2b6c8761dcbe21870c8699d246cbf7435d: Status 404 returned error can't find the container with id 84a7ce7aa1831cffd9c6817731037e2b6c8761dcbe21870c8699d246cbf7435d
Jul 13 18:32:01.545461 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.545444    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fblxn"
Jul 13 18:32:01.551641 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.551621    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c5db7"
Jul 13 18:32:01.555688 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.555666    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh"
Jul 13 18:32:01.563642 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:01.563614    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108324b4_322a_4ba1_9b6a_16e3ba7fea94.slice/crio-9291b7d903dba207574207808fe9038d5c817984206ca0d866e883e043388866 WatchSource:0}: Error finding container 9291b7d903dba207574207808fe9038d5c817984206ca0d866e883e043388866: Status 404 returned error can't find the container with id 9291b7d903dba207574207808fe9038d5c817984206ca0d866e883e043388866
Jul 13 18:32:01.569384 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:01.569363    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb44eda8_27b8_4e19_a0e7_6e3ffa934de6.slice/crio-8e8f9e7eb7295161d5a9bc374858dc6dbb5e503afd66f69625186ad6fceb4be2 WatchSource:0}: Error finding container 8e8f9e7eb7295161d5a9bc374858dc6dbb5e503afd66f69625186ad6fceb4be2: Status 404 returned error can't find the container with id 8e8f9e7eb7295161d5a9bc374858dc6dbb5e503afd66f69625186ad6fceb4be2
Jul 13 18:32:01.824841 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.824755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:01.824996 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.824886    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:01.824996 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.824941    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret podName:8fe11421-9613-4ca6-8c39-c44b3682f360 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:02.824921755 +0000 UTC m=+3.956210195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret") pod "global-pull-secret-syncer-ffsc6" (UID: "8fe11421-9613-4ca6-8c39-c44b3682f360") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:01.914406 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:01.914370    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod588f587a0ae505f4fc590f0ee32209fe.slice/crio-f85fa02f8b74effbb95c20e17ebfc41cc0da7f29471a12c784cecb9a0294e9f5 WatchSource:0}: Error finding container f85fa02f8b74effbb95c20e17ebfc41cc0da7f29471a12c784cecb9a0294e9f5: Status 404 returned error can't find the container with id f85fa02f8b74effbb95c20e17ebfc41cc0da7f29471a12c784cecb9a0294e9f5
Jul 13 18:32:01.925542 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:01.925517    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:01.925650 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.925627    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:01.925706 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:01.925667    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs podName:698c185e-1b6a-43a7-97fb-725ad56c0962 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:02.925654413 +0000 UTC m=+4.056942853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs") pod "network-metrics-daemon-zvvkh" (UID: "698c185e-1b6a-43a7-97fb-725ad56c0962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:02.026235 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.026199    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchnm\" (UniqueName: \"kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm\") pod \"network-check-target-blbvt\" (UID: \"99b3b7e2-f203-4325-945a-214b3a386b4a\") " pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:02.026440 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:02.026385    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:32:02.026440 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:02.026411    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:32:02.026440 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:02.026425    2569 projected.go:196] Error preparing data for projected volume kube-api-access-bchnm for pod openshift-network-diagnostics/network-check-target-blbvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:02.026619 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:02.026478    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm podName:99b3b7e2-f203-4325-945a-214b3a386b4a nodeName:}" failed. No retries permitted until 2026-07-13 18:32:03.026462566 +0000 UTC m=+4.157751005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bchnm" (UniqueName: "kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm") pod "network-check-target-blbvt" (UID: "99b3b7e2-f203-4325-945a-214b3a386b4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:02.265402 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.265264    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 18:27:01 +0000 UTC" deadline="2027-12-10 13:16:21.812330983 +0000 UTC"
Jul 13 18:32:02.265402 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.265341    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12354h44m19.546994712s"
Jul 13 18:32:02.360751 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.360701    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" event={"ID":"588f587a0ae505f4fc590f0ee32209fe","Type":"ContainerStarted","Data":"f85fa02f8b74effbb95c20e17ebfc41cc0da7f29471a12c784cecb9a0294e9f5"}
Jul 13 18:32:02.365966 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.365928    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qs9wn" event={"ID":"d153a23e-6499-4764-ad85-666ef8b2dc72","Type":"ContainerStarted","Data":"84a7ce7aa1831cffd9c6817731037e2b6c8761dcbe21870c8699d246cbf7435d"}
Jul 13 18:32:02.378491 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.378449    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-88fdd" event={"ID":"91a5e102-1948-414f-bfc4-a468dfe9f3ba","Type":"ContainerStarted","Data":"d7ac922ec2d7943339d5a74fc5a103fe51a2fe3314ad17d4e68dcc0249df8a0c"}
Jul 13 18:32:02.388118 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.388073    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9fx5s" event={"ID":"edb49c7e-ea32-4018-9f70-6d25d8b680c4","Type":"ContainerStarted","Data":"4b14d5cd2469ff2bfef5921315f13256b3a29e767f3db3b39bee11ef1fb94cb4"}
Jul 13 18:32:02.395196 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.395061    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t85bl" event={"ID":"d10fd769-b61e-4f9f-9a82-25ff768d4533","Type":"ContainerStarted","Data":"c4cb7be59dff817493ee81f2765ffc1c2e9fdfc03c067aa624fa3b8e16865b78"}
Jul 13 18:32:02.401744 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.401694    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m6256" event={"ID":"e27fd358-8fdd-4706-b5b1-f234e710484a","Type":"ContainerStarted","Data":"03e03e69163fbee30cc29da1f3d3577bf5c6309cc072b591f5bca4e47a599efd"}
Jul 13 18:32:02.422459 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.422420    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh" event={"ID":"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6","Type":"ContainerStarted","Data":"8e8f9e7eb7295161d5a9bc374858dc6dbb5e503afd66f69625186ad6fceb4be2"}
Jul 13 18:32:02.426752 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.426661    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c5db7" event={"ID":"108324b4-322a-4ba1-9b6a-16e3ba7fea94","Type":"ContainerStarted","Data":"9291b7d903dba207574207808fe9038d5c817984206ca0d866e883e043388866"}
Jul 13 18:32:02.433193 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.433162    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fblxn" event={"ID":"bf25eded-5569-4d4d-be04-4eb93300fcf2","Type":"ContainerStarted","Data":"4781a66bb0ab72b45edbf77b2a660be9e735ce327e14779d622794c1006d3d65"}
Jul 13 18:32:02.437610 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.437582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" event={"ID":"10a8e7a5-f7f3-4b20-847a-f8563ff427c7","Type":"ContainerStarted","Data":"79dabf9ed273675a0f4871c885b4d88e603cbf85898f7a2dfb774c5d4fddb9d8"}
Jul 13 18:32:02.832861 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.832771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:02.833076 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:02.832938    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:02.833076 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:02.833004    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret podName:8fe11421-9613-4ca6-8c39-c44b3682f360 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:04.832983713 +0000 UTC m=+5.964272171 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret") pod "global-pull-secret-syncer-ffsc6" (UID: "8fe11421-9613-4ca6-8c39-c44b3682f360") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:02.933879 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:02.933837    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:02.934061 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:02.934018    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:02.934135 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:02.934087    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs podName:698c185e-1b6a-43a7-97fb-725ad56c0962 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:04.934067002 +0000 UTC m=+6.065355461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs") pod "network-metrics-daemon-zvvkh" (UID: "698c185e-1b6a-43a7-97fb-725ad56c0962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:03.036207 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:03.035047    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchnm\" (UniqueName: \"kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm\") pod \"network-check-target-blbvt\" (UID: \"99b3b7e2-f203-4325-945a-214b3a386b4a\") " pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:03.036207 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:03.035231    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:32:03.036207 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:03.035257    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:32:03.036207 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:03.035271    2569 projected.go:196] Error preparing data for projected volume kube-api-access-bchnm for pod openshift-network-diagnostics/network-check-target-blbvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:03.036207 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:03.035335    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm podName:99b3b7e2-f203-4325-945a-214b3a386b4a nodeName:}" failed. No retries permitted until 2026-07-13 18:32:05.035316193 +0000 UTC m=+6.166604633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bchnm" (UniqueName: "kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm") pod "network-check-target-blbvt" (UID: "99b3b7e2-f203-4325-945a-214b3a386b4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:03.266124 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:03.265949    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 18:27:01 +0000 UTC" deadline="2028-02-15 01:12:22.782382828 +0000 UTC"
Jul 13 18:32:03.266124 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:03.265994    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13950h40m19.516394515s"
Jul 13 18:32:03.344830 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:03.344800    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:03.345019 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:03.344939    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:03.345369 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:03.345346    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:03.345473 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:03.345452    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:03.347626 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:03.347601    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:03.347780 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:03.347716    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:04.853149 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:04.853111    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:04.853663 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:04.853304    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:04.853663 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:04.853370    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret podName:8fe11421-9613-4ca6-8c39-c44b3682f360 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:08.853350388 +0000 UTC m=+9.984638831 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret") pod "global-pull-secret-syncer-ffsc6" (UID: "8fe11421-9613-4ca6-8c39-c44b3682f360") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:04.954231 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:04.954196    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:04.954412 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:04.954388    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:04.954490 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:04.954453    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs podName:698c185e-1b6a-43a7-97fb-725ad56c0962 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:08.954433834 +0000 UTC m=+10.085722280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs") pod "network-metrics-daemon-zvvkh" (UID: "698c185e-1b6a-43a7-97fb-725ad56c0962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:05.055466 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:05.055430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchnm\" (UniqueName: \"kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm\") pod \"network-check-target-blbvt\" (UID: \"99b3b7e2-f203-4325-945a-214b3a386b4a\") " pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:05.055703 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:05.055603    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:32:05.055703 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:05.055626    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:32:05.055703 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:05.055636    2569 projected.go:196] Error preparing data for projected volume kube-api-access-bchnm for pod openshift-network-diagnostics/network-check-target-blbvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:05.055855 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:05.055704    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm podName:99b3b7e2-f203-4325-945a-214b3a386b4a nodeName:}" failed. No retries permitted until 2026-07-13 18:32:09.055683585 +0000 UTC m=+10.186972028 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bchnm" (UniqueName: "kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm") pod "network-check-target-blbvt" (UID: "99b3b7e2-f203-4325-945a-214b3a386b4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:05.347857 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:05.346960    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:05.347857 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:05.347109    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:05.347857 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:05.347472    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:05.347857 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:05.347559    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:05.347857 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:05.347602    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:05.347857 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:05.347667    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:07.345234 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:07.345164    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:07.345606 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:07.345164    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:07.345606 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:07.345283    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:07.345606 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:07.345348    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:07.345606 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:07.345471    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:07.345606 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:07.345567    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:08.886843 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:08.886809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:08.887285 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:08.886983    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:08.887285 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:08.887055    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret podName:8fe11421-9613-4ca6-8c39-c44b3682f360 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:16.887033452 +0000 UTC m=+18.018321906 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret") pod "global-pull-secret-syncer-ffsc6" (UID: "8fe11421-9613-4ca6-8c39-c44b3682f360") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:08.987887 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:08.987852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:08.988105 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:08.988076    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:08.988231 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:08.988162    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs podName:698c185e-1b6a-43a7-97fb-725ad56c0962 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:16.988143271 +0000 UTC m=+18.119431713 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs") pod "network-metrics-daemon-zvvkh" (UID: "698c185e-1b6a-43a7-97fb-725ad56c0962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:09.089081 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:09.089044    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchnm\" (UniqueName: \"kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm\") pod \"network-check-target-blbvt\" (UID: \"99b3b7e2-f203-4325-945a-214b3a386b4a\") " pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:09.089702 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:09.089249    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:32:09.089702 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:09.089274    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:32:09.089702 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:09.089287    2569 projected.go:196] Error preparing data for projected volume kube-api-access-bchnm for pod openshift-network-diagnostics/network-check-target-blbvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:09.089702 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:09.089337    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm podName:99b3b7e2-f203-4325-945a-214b3a386b4a nodeName:}" failed. No retries permitted until 2026-07-13 18:32:17.089319049 +0000 UTC m=+18.220607486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bchnm" (UniqueName: "kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm") pod "network-check-target-blbvt" (UID: "99b3b7e2-f203-4325-945a-214b3a386b4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:09.345731 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:09.345309    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:09.345731 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:09.345337    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:09.345731 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:09.345307    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:09.345731 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:09.345395    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:09.345731 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:09.345490    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:09.345731 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:09.345578    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:11.345393 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:11.345155    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:11.345858 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:11.345445    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:11.345858 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:11.345249    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:11.345858 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:11.345534    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:11.345858 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:11.345196    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:11.345858 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:11.345580    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:13.344613 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:13.344586    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:13.345029 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:13.344586    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:13.345029 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:13.344717    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:13.345029 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:13.344726    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:13.345029 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:13.344837    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:13.345029 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:13.344920    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:15.344194 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:15.344158    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:15.344619 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:15.344169    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:15.344619 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:15.344298    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:15.344619 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:15.344316    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:15.344619 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:15.344442    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:15.344619 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:15.344544    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:16.948827 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:16.948783    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:16.949366 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:16.948965    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:16.949366 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:16.949051    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret podName:8fe11421-9613-4ca6-8c39-c44b3682f360 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:32.949029773 +0000 UTC m=+34.080318212 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret") pod "global-pull-secret-syncer-ffsc6" (UID: "8fe11421-9613-4ca6-8c39-c44b3682f360") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:17.050057 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:17.050028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:17.050243 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:17.050155    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:17.050243 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:17.050218    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs podName:698c185e-1b6a-43a7-97fb-725ad56c0962 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:33.050196986 +0000 UTC m=+34.181485431 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs") pod "network-metrics-daemon-zvvkh" (UID: "698c185e-1b6a-43a7-97fb-725ad56c0962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:17.150757 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:17.150718    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchnm\" (UniqueName: \"kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm\") pod \"network-check-target-blbvt\" (UID: \"99b3b7e2-f203-4325-945a-214b3a386b4a\") " pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:17.150920 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:17.150902    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:32:17.150972 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:17.150926    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:32:17.150972 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:17.150939    2569 projected.go:196] Error preparing data for projected volume kube-api-access-bchnm for pod openshift-network-diagnostics/network-check-target-blbvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:17.151032 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:17.150995    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm podName:99b3b7e2-f203-4325-945a-214b3a386b4a nodeName:}" failed. No retries permitted until 2026-07-13 18:32:33.150979384 +0000 UTC m=+34.282267823 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bchnm" (UniqueName: "kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm") pod "network-check-target-blbvt" (UID: "99b3b7e2-f203-4325-945a-214b3a386b4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:17.344533 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:17.344496    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:17.344533 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:17.344530    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:17.344838 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:17.344502    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:17.344838 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:17.344637    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:17.344838 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:17.344733    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:17.344838 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:17.344820    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:19.346012 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:19.345974    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:19.346451 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:19.346068    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:19.346451 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:19.346086    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:19.346451 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:19.346117    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:19.346451 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:19.346195    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:19.346451 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:19.346284    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:21.345283 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:21.344835    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:21.345283 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:21.344835    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:21.345283 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:21.344962    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:21.345283 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:21.344837    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:21.345283 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:21.345033    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:21.345283 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:21.345143    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:22.499465 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:22.499428    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m6256" event={"ID":"e27fd358-8fdd-4706-b5b1-f234e710484a","Type":"ContainerStarted","Data":"45230a740d3ef0c6ce41da7c4f7f34fab01c9e5ca5056b73083ccc2f59cca75d"}
Jul 13 18:32:22.501096 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:22.501070    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c5db7" event={"ID":"108324b4-322a-4ba1-9b6a-16e3ba7fea94","Type":"ContainerStarted","Data":"0c6aab80b9ebf5582095be9772a0c94481603527bb487bbf1e7551744640a176"}
Jul 13 18:32:22.503797 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:22.503777    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:32:22.504233 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:22.504129    2569 generic.go:356] "Generic (PLEG): container finished" podID="10a8e7a5-f7f3-4b20-847a-f8563ff427c7" containerID="2ce5f2b0f535f99d01b4eed6729b594f10cf0fd48f93df0fa822c6cace12f021" exitCode=1
Jul 13 18:32:22.504233 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:22.504193    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" event={"ID":"10a8e7a5-f7f3-4b20-847a-f8563ff427c7","Type":"ContainerDied","Data":"2ce5f2b0f535f99d01b4eed6729b594f10cf0fd48f93df0fa822c6cace12f021"}
Jul 13 18:32:22.504368 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:22.504283    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" event={"ID":"10a8e7a5-f7f3-4b20-847a-f8563ff427c7","Type":"ContainerStarted","Data":"b6b53c9f20bfce20873270c6ed9187a1369cb08e44cfdee0851731df5e13d777"}
Jul 13 18:32:22.507092 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:22.507058    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" event={"ID":"48897f40f8f0e850d106dd2f49700b6a","Type":"ContainerStarted","Data":"4037215d567806e4f53a411ec5debd33c28f3aeed6a0b5dd4bd140bdb0afab6f"}
Jul 13 18:32:22.526399 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:22.526352    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m6256" podStartSLOduration=1.8952355669999998 podStartE2EDuration="22.526333785s" totalImagesPullingTime="20.631098218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:22.526104871 +0000 UTC m=+23.657393332" watchObservedRunningTime="2026-07-13 18:32:22.526333785 +0000 UTC m=+23.657622246"
Jul 13 18:32:22.571076 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:22.570997    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-c5db7" podStartSLOduration=1.951975317 podStartE2EDuration="22.570977479s" totalImagesPullingTime="20.619002162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:22.550572245 +0000 UTC m=+23.681860705" watchObservedRunningTime="2026-07-13 18:32:22.570977479 +0000 UTC m=+23.702265939"
Jul 13 18:32:22.571513 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:22.571475    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" podStartSLOduration=21.571463519 podStartE2EDuration="21.571463519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:22.570501084 +0000 UTC m=+23.701789546" watchObservedRunningTime="2026-07-13 18:32:22.571463519 +0000 UTC m=+23.702751980"
Jul 13 18:32:23.344433 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.344115    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:23.344590 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.344114    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:23.344590 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:23.344512    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:23.344590 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.344174    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:23.344807 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:23.344690    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:23.344807 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:23.344569    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:23.510274 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.510233    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t85bl" event={"ID":"d10fd769-b61e-4f9f-9a82-25ff768d4533","Type":"ContainerStarted","Data":"1c575fc036967fb2ead02007d12369463b90cf9467789bebd6cbfa9dd4ed6322"}
Jul 13 18:32:23.511637 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.511614    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh" event={"ID":"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6","Type":"ContainerStarted","Data":"d080e9937055aa02f65630004192016550fcedc15332e0a97ed64e28012e6f21"}
Jul 13 18:32:23.513018 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.512994    2569 generic.go:356] "Generic (PLEG): container finished" podID="bf25eded-5569-4d4d-be04-4eb93300fcf2" containerID="6ca9cac7162ef81835d9168cff969db4f51135b313e429dc7979256c7dabb34b" exitCode=0
Jul 13 18:32:23.513139 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.513074    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fblxn" event={"ID":"bf25eded-5569-4d4d-be04-4eb93300fcf2","Type":"ContainerDied","Data":"6ca9cac7162ef81835d9168cff969db4f51135b313e429dc7979256c7dabb34b"}
Jul 13 18:32:23.515912 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.515895    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:32:23.516283 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.516261    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" event={"ID":"10a8e7a5-f7f3-4b20-847a-f8563ff427c7","Type":"ContainerStarted","Data":"a009a6b9d5f4cece0ad14a41b9a1f951a8bc80929ad6f003adf91353c5c9f747"}
Jul 13 18:32:23.516348 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.516337    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" event={"ID":"10a8e7a5-f7f3-4b20-847a-f8563ff427c7","Type":"ContainerStarted","Data":"ee3a4e9da6319addaa59b2b31a770bcf6946e25250f052e869bfd0eed8181736"}
Jul 13 18:32:23.516403 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.516390    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" event={"ID":"10a8e7a5-f7f3-4b20-847a-f8563ff427c7","Type":"ContainerStarted","Data":"879df8e64bee1730d91ca1bc24486fe13ae91ffc18ae367fc6f1233764fae63c"}
Jul 13 18:32:23.516455 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.516443    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" event={"ID":"10a8e7a5-f7f3-4b20-847a-f8563ff427c7","Type":"ContainerStarted","Data":"c9c0ec996d24291fe6860acb1bb08ba0ae313c2ac32ee1896014cda4ae283ec6"}
Jul 13 18:32:23.517690 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.517670    2569 generic.go:356] "Generic (PLEG): container finished" podID="588f587a0ae505f4fc590f0ee32209fe" containerID="531756c863545f185993bc8920eabef1aba8b7f8b0055ebdf1a07e23c1491086" exitCode=0
Jul 13 18:32:23.517784 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.517702    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" event={"ID":"588f587a0ae505f4fc590f0ee32209fe","Type":"ContainerDied","Data":"531756c863545f185993bc8920eabef1aba8b7f8b0055ebdf1a07e23c1491086"}
Jul 13 18:32:23.519233 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.519202    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qs9wn" event={"ID":"d153a23e-6499-4764-ad85-666ef8b2dc72","Type":"ContainerStarted","Data":"99314b9b2d915495a32c385e946d614579d77267c0384bd64396a503055fc74e"}
Jul 13 18:32:23.520586 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.520566    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-88fdd" event={"ID":"91a5e102-1948-414f-bfc4-a468dfe9f3ba","Type":"ContainerStarted","Data":"1b9e33e6d7078079d8abd0d64120cf98110f37b6e7785bcd41d1798b4e00d9d9"}
Jul 13 18:32:23.521901 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.521883    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9fx5s" event={"ID":"edb49c7e-ea32-4018-9f70-6d25d8b680c4","Type":"ContainerStarted","Data":"8a24548263f51ed0a1d621b430c7fe16ba73910a8a72fffa3cdeb83cae0d20a8"}
Jul 13 18:32:23.524538 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.524504    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t85bl" podStartSLOduration=2.926810551 podStartE2EDuration="23.524492725s" totalImagesPullingTime="20.597682174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:23.524305784 +0000 UTC m=+24.655594234" watchObservedRunningTime="2026-07-13 18:32:23.524492725 +0000 UTC m=+24.655781184"
Jul 13 18:32:23.539212 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.539173    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9fx5s" podStartSLOduration=7.349302386 podStartE2EDuration="23.539162765s" totalImagesPullingTime="16.189860379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:23.538719347 +0000 UTC m=+24.670007818" watchObservedRunningTime="2026-07-13 18:32:23.539162765 +0000 UTC m=+24.670451284"
Jul 13 18:32:23.554006 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.553971    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qs9wn" podStartSLOduration=2.967970679 podStartE2EDuration="23.553959337s" totalImagesPullingTime="20.585988658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:23.553919646 +0000 UTC m=+24.685208100" watchObservedRunningTime="2026-07-13 18:32:23.553959337 +0000 UTC m=+24.685247792"
Jul 13 18:32:23.588117 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:23.588072    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-88fdd" podStartSLOduration=2.979039701 podStartE2EDuration="23.588057345s" totalImagesPullingTime="20.609017644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:23.587795416 +0000 UTC m=+24.719083876" watchObservedRunningTime="2026-07-13 18:32:23.588057345 +0000 UTC m=+24.719345804"
Jul 13 18:32:24.293977 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:24.293954    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 18:32:24.525248 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:24.525214    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh" event={"ID":"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6","Type":"ContainerStarted","Data":"42e3b2bea99c90012e8162c0cf8fbdb77b6f587d9ae213953fe8ef335622413d"}
Jul 13 18:32:24.526975 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:24.526945    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" event={"ID":"588f587a0ae505f4fc590f0ee32209fe","Type":"ContainerStarted","Data":"fad022bd57e92e342feeb941e479284935c99d139c1a2fd79610ed1eaf7309ba"}
Jul 13 18:32:24.543599 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:24.543561    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" podStartSLOduration=23.543547543 podStartE2EDuration="23.543547543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:24.543471637 +0000 UTC m=+25.674760096" watchObservedRunningTime="2026-07-13 18:32:24.543547543 +0000 UTC m=+25.674835997"
Jul 13 18:32:24.935434 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:24.935351    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:24.936000 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:24.935980    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:25.042857 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:25.042826    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:25.043394 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:25.043379    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9fx5s"
Jul 13 18:32:25.283833 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:25.283724    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T18:32:24.293972672Z","UUID":"2f5410c1-dac3-4438-acc2-a725630791ba","Handler":null,"Name":"","Endpoint":""}
Jul 13 18:32:25.285421 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:25.285400    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 18:32:25.285548 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:25.285442    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 18:32:25.344248 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:25.344221    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:25.344361 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:25.344226    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:25.344361 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:25.344343    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:25.344516 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:25.344423    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:25.344516 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:25.344493    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:25.344612 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:25.344583    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:25.533254 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:25.533231    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:32:25.533781 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:25.533675    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" event={"ID":"10a8e7a5-f7f3-4b20-847a-f8563ff427c7","Type":"ContainerStarted","Data":"499e95cce0f9c40d26077b79898429d1811d341a708701ba66fc7df8909578e1"}
Jul 13 18:32:26.538764 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:26.538714    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh" event={"ID":"cb44eda8-27b8-4e19-a0e7-6e3ffa934de6","Type":"ContainerStarted","Data":"2e95bfb32331a3361c28961c42222f4a35bbb8e84497bda5526892450fca34fd"}
Jul 13 18:32:26.560951 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:26.560902    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7h5xh" podStartSLOduration=3.402882621 podStartE2EDuration="26.560885399s" totalImagesPullingTime="23.158002778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:26.560799598 +0000 UTC m=+27.692088057" watchObservedRunningTime="2026-07-13 18:32:26.560885399 +0000 UTC m=+27.692173859"
Jul 13 18:32:27.344577 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:27.344497    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:27.344577 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:27.344522    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:27.344810 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:27.344509    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:27.344810 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:27.344633    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:27.344810 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:27.344729    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:27.344970 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:27.344815    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:29.345771 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.345575    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:29.346541 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.345643    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:29.346541 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:29.345835    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:29.346541 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.345688    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:29.346541 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:29.345909    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:29.346541 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:29.346005    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:29.546971 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.546939    2569 generic.go:356] "Generic (PLEG): container finished" podID="bf25eded-5569-4d4d-be04-4eb93300fcf2" containerID="b61cd2d7b12148eda0729d73fa682c8ed0c1ad998c5c7585bf5eeaab8c182d53" exitCode=0
Jul 13 18:32:29.547176 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.547024    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fblxn" event={"ID":"bf25eded-5569-4d4d-be04-4eb93300fcf2","Type":"ContainerDied","Data":"b61cd2d7b12148eda0729d73fa682c8ed0c1ad998c5c7585bf5eeaab8c182d53"}
Jul 13 18:32:29.550681 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.550645    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:32:29.551050 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.550999    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" event={"ID":"10a8e7a5-f7f3-4b20-847a-f8563ff427c7","Type":"ContainerStarted","Data":"817f9064891d0c75e1ee5e95dbf59fe4405392daf26faafd9940fe5fd1250f6b"}
Jul 13 18:32:29.551266 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.551249    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:29.551375 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.551286    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:29.551451 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.551434    2569 scope.go:117] "RemoveContainer" containerID="2ce5f2b0f535f99d01b4eed6729b594f10cf0fd48f93df0fa822c6cace12f021"
Jul 13 18:32:29.568450 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:29.568431    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:30.557273 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.557101    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:32:30.557688 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.557565    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" event={"ID":"10a8e7a5-f7f3-4b20-847a-f8563ff427c7","Type":"ContainerStarted","Data":"07fa2bd87c1ac16bad5754db97ae2e36a555e4f7e0781be8e566a9f3020d24f4"}
Jul 13 18:32:30.557761 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.557747    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:30.574966 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.574940    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:32:30.656210 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.656100    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn" podStartSLOduration=10.060217766 podStartE2EDuration="30.656081944s" totalImagesPullingTime="20.595864178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:30.619086307 +0000 UTC m=+31.750374767" watchObservedRunningTime="2026-07-13 18:32:30.656081944 +0000 UTC m=+31.787370405"
Jul 13 18:32:30.681861 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.681829    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ffsc6"]
Jul 13 18:32:30.681988 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.681969    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:30.682078 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:30.682057    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:30.688399 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.688318    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zvvkh"]
Jul 13 18:32:30.688525 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.688494    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:30.688601 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:30.688585    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:30.689149 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.689126    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-blbvt"]
Jul 13 18:32:30.689255 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:30.689228    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:30.689318 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:30.689303    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:31.562102 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:31.562072    2569 generic.go:356] "Generic (PLEG): container finished" podID="bf25eded-5569-4d4d-be04-4eb93300fcf2" containerID="2bd11b2580c19a31b392ab509b7f45dd663d1c5cfed90f4abbc67cb3b802881f" exitCode=0
Jul 13 18:32:31.562482 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:31.562153    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fblxn" event={"ID":"bf25eded-5569-4d4d-be04-4eb93300fcf2","Type":"ContainerDied","Data":"2bd11b2580c19a31b392ab509b7f45dd663d1c5cfed90f4abbc67cb3b802881f"}
Jul 13 18:32:32.344599 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:32.344572    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:32.344771 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:32.344698    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:32.344771 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:32.344742    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:32.344883 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:32.344781    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:32.344883 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:32.344863    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:32.344941 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:32.344925    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:32.566143 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:32.566114    2569 generic.go:356] "Generic (PLEG): container finished" podID="bf25eded-5569-4d4d-be04-4eb93300fcf2" containerID="30ec81dec9b2ae416026fa76360c9d9405a420c750a1eba4c0aa65680155bf59" exitCode=0
Jul 13 18:32:32.566468 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:32.566188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fblxn" event={"ID":"bf25eded-5569-4d4d-be04-4eb93300fcf2","Type":"ContainerDied","Data":"30ec81dec9b2ae416026fa76360c9d9405a420c750a1eba4c0aa65680155bf59"}
Jul 13 18:32:32.968930 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:32.968900    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:32.969087 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:32.969018    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:32.969087 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:32.969066    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret podName:8fe11421-9613-4ca6-8c39-c44b3682f360 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:04.969052897 +0000 UTC m=+66.100341336 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret") pod "global-pull-secret-syncer-ffsc6" (UID: "8fe11421-9613-4ca6-8c39-c44b3682f360") : object "kube-system"/"original-pull-secret" not registered
Jul 13 18:32:33.069884 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:33.069814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:33.070020 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:33.069928    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:33.070020 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:33.069975    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs podName:698c185e-1b6a-43a7-97fb-725ad56c0962 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:05.069963294 +0000 UTC m=+66.201251732 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs") pod "network-metrics-daemon-zvvkh" (UID: "698c185e-1b6a-43a7-97fb-725ad56c0962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 18:32:33.170750 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:33.170722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchnm\" (UniqueName: \"kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm\") pod \"network-check-target-blbvt\" (UID: \"99b3b7e2-f203-4325-945a-214b3a386b4a\") " pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:33.170901 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:33.170844    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 18:32:33.170901 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:33.170859    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 18:32:33.170901 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:33.170867    2569 projected.go:196] Error preparing data for projected volume kube-api-access-bchnm for pod openshift-network-diagnostics/network-check-target-blbvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:33.171010 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:33.170910    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm podName:99b3b7e2-f203-4325-945a-214b3a386b4a nodeName:}" failed. No retries permitted until 2026-07-13 18:33:05.170897809 +0000 UTC m=+66.302186246 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bchnm" (UniqueName: "kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm") pod "network-check-target-blbvt" (UID: "99b3b7e2-f203-4325-945a-214b3a386b4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 18:32:34.344959 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:34.344930    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:34.344959 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:34.344948    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:34.345460 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:34.344958    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:34.345460 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:34.345046    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blbvt" podUID="99b3b7e2-f203-4325-945a-214b3a386b4a"
Jul 13 18:32:34.345460 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:34.345122    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffsc6" podUID="8fe11421-9613-4ca6-8c39-c44b3682f360"
Jul 13 18:32:34.345460 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:34.345213    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zvvkh" podUID="698c185e-1b6a-43a7-97fb-725ad56c0962"
Jul 13 18:32:34.893156 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:34.893112    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:35.000484 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:35.000380    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:35.529814 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:35.529777    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:35.589888 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:35.589800    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:36.194776 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.194528    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeReady"
Jul 13 18:32:36.194948 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.194883    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 18:32:36.291420 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.290503    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-s8dct"]
Jul 13 18:32:36.295163 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.295137    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nlnzn"]
Jul 13 18:32:36.295287 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.295257    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.298500 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.298482    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nlnzn"
Jul 13 18:32:36.300381 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.300365    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tnbnw\""
Jul 13 18:32:36.301737 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.301717    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 18:32:36.302870 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.302507    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 18:32:36.304271 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.303370    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 18:32:36.304271 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.303444    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 18:32:36.304271 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.304199    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-phpw8\""
Jul 13 18:32:36.304271 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.304209    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 18:32:36.320715 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.320688    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s8dct"]
Jul 13 18:32:36.322230 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.322206    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nlnzn"]
Jul 13 18:32:36.345006 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.344981    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:32:36.345006 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.344997    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:32:36.345178 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.344988    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:32:36.349563 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.349421    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 18:32:36.349563 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.349441    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 18:32:36.349563 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.349463    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 18:32:36.349563 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.349426    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 18:32:36.349829 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.349764    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bwd4z\""
Jul 13 18:32:36.349927 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.349912    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ql2kw\""
Jul 13 18:32:36.395244 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.395216    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4lr8\" (UniqueName: \"kubernetes.io/projected/d1729c34-3c2c-4fa7-a94e-f7628fad363e-kube-api-access-d4lr8\") pod \"ingress-canary-nlnzn\" (UID: \"d1729c34-3c2c-4fa7-a94e-f7628fad363e\") " pod="openshift-ingress-canary/ingress-canary-nlnzn"
Jul 13 18:32:36.395373 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.395273    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.395373 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.395300    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1729c34-3c2c-4fa7-a94e-f7628fad363e-cert\") pod \"ingress-canary-nlnzn\" (UID: \"d1729c34-3c2c-4fa7-a94e-f7628fad363e\") " pod="openshift-ingress-canary/ingress-canary-nlnzn"
Jul 13 18:32:36.395373 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.395351    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-tmp-dir\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.395515 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.395378    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsgwv\" (UniqueName: \"kubernetes.io/projected/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-kube-api-access-vsgwv\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.395515 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.395410    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-config-volume\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.496483 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.496450    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-tmp-dir\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.496483 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.496483    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsgwv\" (UniqueName: \"kubernetes.io/projected/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-kube-api-access-vsgwv\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.496753 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.496519    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-config-volume\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.496753 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.496592    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4lr8\" (UniqueName: \"kubernetes.io/projected/d1729c34-3c2c-4fa7-a94e-f7628fad363e-kube-api-access-d4lr8\") pod \"ingress-canary-nlnzn\" (UID: \"d1729c34-3c2c-4fa7-a94e-f7628fad363e\") " pod="openshift-ingress-canary/ingress-canary-nlnzn"
Jul 13 18:32:36.496753 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.496627    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.496753 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.496642    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1729c34-3c2c-4fa7-a94e-f7628fad363e-cert\") pod \"ingress-canary-nlnzn\" (UID: \"d1729c34-3c2c-4fa7-a94e-f7628fad363e\") " pod="openshift-ingress-canary/ingress-canary-nlnzn"
Jul 13 18:32:36.496960 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.496860    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-tmp-dir\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.497351 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.497318    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-config-volume\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.497351 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:36.497345    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 18:32:36.497527 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:36.497412    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls podName:60f2abdb-0f87-4d7b-abdd-b9dfef4eb951 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:36.997393141 +0000 UTC m=+38.128681579 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls") pod "dns-default-s8dct" (UID: "60f2abdb-0f87-4d7b-abdd-b9dfef4eb951") : secret "dns-default-metrics-tls" not found
Jul 13 18:32:36.501058 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.501039    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1729c34-3c2c-4fa7-a94e-f7628fad363e-cert\") pod \"ingress-canary-nlnzn\" (UID: \"d1729c34-3c2c-4fa7-a94e-f7628fad363e\") " pod="openshift-ingress-canary/ingress-canary-nlnzn"
Jul 13 18:32:36.509371 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.509348    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4lr8\" (UniqueName: \"kubernetes.io/projected/d1729c34-3c2c-4fa7-a94e-f7628fad363e-kube-api-access-d4lr8\") pod \"ingress-canary-nlnzn\" (UID: \"d1729c34-3c2c-4fa7-a94e-f7628fad363e\") " pod="openshift-ingress-canary/ingress-canary-nlnzn"
Jul 13 18:32:36.510305 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.510284    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsgwv\" (UniqueName: \"kubernetes.io/projected/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-kube-api-access-vsgwv\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.612158 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.612124    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nlnzn"
Jul 13 18:32:36.999537 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:36.999507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:36.999714 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:36.999618    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 18:32:36.999714 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:36.999702    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls podName:60f2abdb-0f87-4d7b-abdd-b9dfef4eb951 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:37.999686377 +0000 UTC m=+39.130974819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls") pod "dns-default-s8dct" (UID: "60f2abdb-0f87-4d7b-abdd-b9dfef4eb951") : secret "dns-default-metrics-tls" not found
Jul 13 18:32:37.098860 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:37.098728    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nlnzn"]
Jul 13 18:32:37.102337 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:37.102308    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1729c34_3c2c_4fa7_a94e_f7628fad363e.slice/crio-72141b796b060a4d366b1d8d9bbc1e97eb9febaca160d98f9bfe7ca61c51ec5f WatchSource:0}: Error finding container 72141b796b060a4d366b1d8d9bbc1e97eb9febaca160d98f9bfe7ca61c51ec5f: Status 404 returned error can't find the container with id 72141b796b060a4d366b1d8d9bbc1e97eb9febaca160d98f9bfe7ca61c51ec5f
Jul 13 18:32:37.580932 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:37.580896    2569 generic.go:356] "Generic (PLEG): container finished" podID="bf25eded-5569-4d4d-be04-4eb93300fcf2" containerID="4c390537fad1d294135eb760d49cf5d84c2515876b6fa846e556825686274caa" exitCode=0
Jul 13 18:32:37.581111 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:37.580934    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fblxn" event={"ID":"bf25eded-5569-4d4d-be04-4eb93300fcf2","Type":"ContainerDied","Data":"4c390537fad1d294135eb760d49cf5d84c2515876b6fa846e556825686274caa"}
Jul 13 18:32:37.582139 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:37.582103    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nlnzn" event={"ID":"d1729c34-3c2c-4fa7-a94e-f7628fad363e","Type":"ContainerStarted","Data":"72141b796b060a4d366b1d8d9bbc1e97eb9febaca160d98f9bfe7ca61c51ec5f"}
Jul 13 18:32:38.006453 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:38.006424    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:38.007130 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:38.006539    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 18:32:38.007130 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:38.006606    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls podName:60f2abdb-0f87-4d7b-abdd-b9dfef4eb951 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:40.006587911 +0000 UTC m=+41.137876352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls") pod "dns-default-s8dct" (UID: "60f2abdb-0f87-4d7b-abdd-b9dfef4eb951") : secret "dns-default-metrics-tls" not found
Jul 13 18:32:38.587824 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:38.587792    2569 generic.go:356] "Generic (PLEG): container finished" podID="bf25eded-5569-4d4d-be04-4eb93300fcf2" containerID="812eb702c9290c29697fe45803dd061b589b607a4c53ac409a7a4712351f05b5" exitCode=0
Jul 13 18:32:38.587978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:38.587829    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fblxn" event={"ID":"bf25eded-5569-4d4d-be04-4eb93300fcf2","Type":"ContainerDied","Data":"812eb702c9290c29697fe45803dd061b589b607a4c53ac409a7a4712351f05b5"}
Jul 13 18:32:39.594505 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:39.594474    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fblxn" event={"ID":"bf25eded-5569-4d4d-be04-4eb93300fcf2","Type":"ContainerStarted","Data":"a2cdb5591438e4d71e1f28f436ff989c6b7a34cb8b943e9c5b58af58e2365f21"}
Jul 13 18:32:39.596093 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:39.596059    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nlnzn" event={"ID":"d1729c34-3c2c-4fa7-a94e-f7628fad363e","Type":"ContainerStarted","Data":"8ee540596cdf8e6953ef8d71b1ac0e20e4f2d22392dab630ea39c550d6b8032a"}
Jul 13 18:32:39.620258 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:39.620197    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fblxn" podStartSLOduration=7.3241037460000005 podStartE2EDuration="39.620173277s" totalImagesPullingTime="32.296069531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:39.619838108 +0000 UTC m=+40.751126567" watchObservedRunningTime="2026-07-13 18:32:39.620173277 +0000 UTC m=+40.751461738"
Jul 13 18:32:39.636828 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:39.636779    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nlnzn" podStartSLOduration=1.232610148 podStartE2EDuration="3.636759127s" totalImagesPullingTime="2.404148979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:39.636180863 +0000 UTC m=+40.767469324" watchObservedRunningTime="2026-07-13 18:32:39.636759127 +0000 UTC m=+40.768047585"
Jul 13 18:32:39.743258 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:39.743071    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:39.841160 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:39.841073    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:39.941006 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:39.940978    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:40.012728 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:40.012696    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:40.021490 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:40.021466    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:40.021593 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:40.021580    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 18:32:40.021645 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:40.021635    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls podName:60f2abdb-0f87-4d7b-abdd-b9dfef4eb951 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:44.021618893 +0000 UTC m=+45.152907331 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls") pod "dns-default-s8dct" (UID: "60f2abdb-0f87-4d7b-abdd-b9dfef4eb951") : secret "dns-default-metrics-tls" not found
Jul 13 18:32:42.850492 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:42.850450    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:42.909989 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:42.909961    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:44.048726 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:44.048689    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:44.049127 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:44.048797    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 18:32:44.049127 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:44.048851    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls podName:60f2abdb-0f87-4d7b-abdd-b9dfef4eb951 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:52.048836619 +0000 UTC m=+53.180125057 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls") pod "dns-default-s8dct" (UID: "60f2abdb-0f87-4d7b-abdd-b9dfef4eb951") : secret "dns-default-metrics-tls" not found
Jul 13 18:32:46.082253 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.082225    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"]
Jul 13 18:32:46.114209 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.114187    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"]
Jul 13 18:32:46.114353 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.114270    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:46.117028 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.117004    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 13 18:32:46.118180 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.118161    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tgsln\""
Jul 13 18:32:46.119503 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.119485    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 18:32:46.119623 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.119485    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 18:32:46.119623 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.119548    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 13 18:32:46.215673 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.215627    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74598d9c75-tkpgp"]
Jul 13 18:32:46.241597 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.241571    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"]
Jul 13 18:32:46.241747 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.241708    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.244618 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.244589    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gx49k\""
Jul 13 18:32:46.244894 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.244875    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 13 18:32:46.245177 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.245159    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 13 18:32:46.245455 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.245431    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 13 18:32:46.256636 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.256615    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 13 18:32:46.257603 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.257583    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"]
Jul 13 18:32:46.257724 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.257710    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.262685 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.262635    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0b35e240-a9b7-4d64-be6b-bb32082a2073-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:46.262808 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.262787    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:46.262869 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.262821    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kndkc\" (UniqueName: \"kubernetes.io/projected/0b35e240-a9b7-4d64-be6b-bb32082a2073-kube-api-access-kndkc\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:46.263325 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.263304    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 13 18:32:46.263615 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.263597    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2nvn8\""
Jul 13 18:32:46.263876 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.263628    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 13 18:32:46.264065 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.264050    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 13 18:32:46.264177 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.264163    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 13 18:32:46.277739 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.277721    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-597d7847fb-55tbz"]
Jul 13 18:32:46.277875 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.277857    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:32:46.281025 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.281004    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 13 18:32:46.281229 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.281032    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 13 18:32:46.281302 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.281042    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 13 18:32:46.281402 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.281380    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nh4mn\""
Jul 13 18:32:46.301805 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.301774    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74598d9c75-tkpgp"]
Jul 13 18:32:46.301805 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.301806    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"]
Jul 13 18:32:46.302017 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.301819    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"]
Jul 13 18:32:46.302017 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.301836    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-597d7847fb-55tbz"]
Jul 13 18:32:46.302017 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.301908    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.307060 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.307042    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 13 18:32:46.307331 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.307318    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 13 18:32:46.307509 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.307488    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 13 18:32:46.307597 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.307492    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 13 18:32:46.307931 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.307790    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gghh4\""
Jul 13 18:32:46.308032 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.308016    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 13 18:32:46.308076 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.308040    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 13 18:32:46.363283 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363218    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-bound-sa-token\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.363283 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363247    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc90ccc1-03cf-49db-bfd4-5458ec6e63a1-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-cndps\" (UID: \"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.363283 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363267    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-trusted-ca\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.363478 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:32:46.363478 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363378    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-image-registry-private-configuration\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.363478 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363402    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqsxs\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-kube-api-access-bqsxs\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.363478 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363440    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0b35e240-a9b7-4d64-be6b-bb32082a2073-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:46.363598 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363484    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6be9563e-101a-40e5-bff8-17e166e7931e-ca-trust-extracted\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.363598 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363505    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-installation-pull-secrets\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.363598 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363538    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc90ccc1-03cf-49db-bfd4-5458ec6e63a1-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-cndps\" (UID: \"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.363598 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b778\" (UniqueName: \"kubernetes.io/projected/3351b763-29c9-477b-b8e7-61dfc79ec616-kube-api-access-9b778\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:32:46.363739 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-registry-certificates\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.363739 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363669    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmzpx\" (UniqueName: \"kubernetes.io/projected/cc90ccc1-03cf-49db-bfd4-5458ec6e63a1-kube-api-access-vmzpx\") pod \"kube-storage-version-migrator-operator-799cb7f958-cndps\" (UID: \"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.363739 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:46.363739 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363727    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kndkc\" (UniqueName: \"kubernetes.io/projected/0b35e240-a9b7-4d64-be6b-bb32082a2073-kube-api-access-kndkc\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:46.363873 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.363742    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.363873 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.363807    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 18:32:46.363873 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.363848    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls podName:0b35e240-a9b7-4d64-be6b-bb32082a2073 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:46.863835512 +0000 UTC m=+47.995123951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hrjpr" (UID: "0b35e240-a9b7-4d64-be6b-bb32082a2073") : secret "cluster-monitoring-operator-tls" not found
Jul 13 18:32:46.364171 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.364154    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0b35e240-a9b7-4d64-be6b-bb32082a2073-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:46.400963 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.400935    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kndkc\" (UniqueName: \"kubernetes.io/projected/0b35e240-a9b7-4d64-be6b-bb32082a2073-kube-api-access-kndkc\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:46.464294 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-trusted-ca\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.464451 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464304    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-default-certificate\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.464451 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464327    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfs8g\" (UniqueName: \"kubernetes.io/projected/f5c0e7af-b62f-4dfc-810f-39223c968ce9-kube-api-access-qfs8g\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.464451 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464414    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-installation-pull-secrets\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.464451 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc90ccc1-03cf-49db-bfd4-5458ec6e63a1-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-cndps\" (UID: \"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.464591 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464468    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b778\" (UniqueName: \"kubernetes.io/projected/3351b763-29c9-477b-b8e7-61dfc79ec616-kube-api-access-9b778\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:32:46.464591 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464510    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:32:46.464591 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464540    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.464591 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464579    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc90ccc1-03cf-49db-bfd4-5458ec6e63a1-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-cndps\" (UID: \"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.464795 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.464795 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.464707    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 18:32:46.464795 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.464718    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 18:32:46.464927 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.464842    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74598d9c75-tkpgp: secret "image-registry-tls" not found
Jul 13 18:32:46.464927 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.464763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.464927 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.464785    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls podName:3351b763-29c9-477b-b8e7-61dfc79ec616 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:46.964762415 +0000 UTC m=+48.096050870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls") pod "cluster-samples-operator-65db858894-dsjk6" (UID: "3351b763-29c9-477b-b8e7-61dfc79ec616") : secret "samples-operator-tls" not found
Jul 13 18:32:46.465075 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.464987    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls podName:6be9563e-101a-40e5-bff8-17e166e7931e nodeName:}" failed. No retries permitted until 2026-07-13 18:32:46.964960635 +0000 UTC m=+48.096249074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls") pod "image-registry-74598d9c75-tkpgp" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e") : secret "image-registry-tls" not found
Jul 13 18:32:46.465075 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465023    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-image-registry-private-configuration\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.465075 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465047    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqsxs\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-kube-api-access-bqsxs\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.465075 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465065    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6be9563e-101a-40e5-bff8-17e166e7931e-ca-trust-extracted\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.465266 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmzpx\" (UniqueName: \"kubernetes.io/projected/cc90ccc1-03cf-49db-bfd4-5458ec6e63a1-kube-api-access-vmzpx\") pod \"kube-storage-version-migrator-operator-799cb7f958-cndps\" (UID: \"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.465266 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465132    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-registry-certificates\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.465266 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465139    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc90ccc1-03cf-49db-bfd4-5458ec6e63a1-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-cndps\" (UID: \"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.465266 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465179    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-bound-sa-token\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.465464 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465328    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-stats-auth\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.465525 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465486    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-trusted-ca\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.465890 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465869    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6be9563e-101a-40e5-bff8-17e166e7931e-ca-trust-extracted\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.465966 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.465869    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-registry-certificates\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.467038 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.467018    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc90ccc1-03cf-49db-bfd4-5458ec6e63a1-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-cndps\" (UID: \"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.467298 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.467280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-installation-pull-secrets\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.467341 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.467295    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-image-registry-private-configuration\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.479870 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.479846    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-bound-sa-token\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.479960 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.479851    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b778\" (UniqueName: \"kubernetes.io/projected/3351b763-29c9-477b-b8e7-61dfc79ec616-kube-api-access-9b778\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:32:46.479960 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.479898    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmzpx\" (UniqueName: \"kubernetes.io/projected/cc90ccc1-03cf-49db-bfd4-5458ec6e63a1-kube-api-access-vmzpx\") pod \"kube-storage-version-migrator-operator-799cb7f958-cndps\" (UID: \"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.480850 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.480834    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqsxs\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-kube-api-access-bqsxs\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.565943 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.565908    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"
Jul 13 18:32:46.566100 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.566078    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-stats-auth\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.566144 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.566112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-default-certificate\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.566144 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.566136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfs8g\" (UniqueName: \"kubernetes.io/projected/f5c0e7af-b62f-4dfc-810f-39223c968ce9-kube-api-access-qfs8g\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.566254 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.566239    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.566304 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.566274    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.566374 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.566355    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 18:32:46.566493 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.566400    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:47.066384559 +0000 UTC m=+48.197672997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : configmap references non-existent config key: service-ca.crt
Jul 13 18:32:46.566493 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.566429    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:47.066411857 +0000 UTC m=+48.197700309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : secret "router-metrics-certs-default" not found
Jul 13 18:32:46.568686 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.568644    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-default-certificate\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.568757 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.568696    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-stats-auth\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.588481 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.588461    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfs8g\" (UniqueName: \"kubernetes.io/projected/f5c0e7af-b62f-4dfc-810f-39223c968ce9-kube-api-access-qfs8g\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:46.595531 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.595509    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:46.716882 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.716840    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:46.725328 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.725305    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps"]
Jul 13 18:32:46.729077 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:32:46.729054    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc90ccc1_03cf_49db_bfd4_5458ec6e63a1.slice/crio-91b3a7493a658a804e465618d85358227bc49e44b5864e34c39e0ab4524cd0d7 WatchSource:0}: Error finding container 91b3a7493a658a804e465618d85358227bc49e44b5864e34c39e0ab4524cd0d7: Status 404 returned error can't find the container with id 91b3a7493a658a804e465618d85358227bc49e44b5864e34c39e0ab4524cd0d7
Jul 13 18:32:46.820881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.820849    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:46.869298 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.869271    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:46.869449 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.869431    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 18:32:46.869518 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.869506    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls podName:0b35e240-a9b7-4d64-be6b-bb32082a2073 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:47.86948582 +0000 UTC m=+49.000774262 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hrjpr" (UID: "0b35e240-a9b7-4d64-be6b-bb32082a2073") : secret "cluster-monitoring-operator-tls" not found
Jul 13 18:32:46.915537 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.915474    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:46.970591 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.970564    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:32:46.970724 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:46.970594    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:46.970724 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.970715    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 18:32:46.970812 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.970719    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 18:32:46.970812 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.970727    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74598d9c75-tkpgp: secret "image-registry-tls" not found
Jul 13 18:32:46.970812 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.970776    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls podName:3351b763-29c9-477b-b8e7-61dfc79ec616 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:47.970761026 +0000 UTC m=+49.102049464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls") pod "cluster-samples-operator-65db858894-dsjk6" (UID: "3351b763-29c9-477b-b8e7-61dfc79ec616") : secret "samples-operator-tls" not found
Jul 13 18:32:46.970812 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:46.970790    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls podName:6be9563e-101a-40e5-bff8-17e166e7931e nodeName:}" failed. No retries permitted until 2026-07-13 18:32:47.970782728 +0000 UTC m=+49.102071166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls") pod "image-registry-74598d9c75-tkpgp" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e") : secret "image-registry-tls" not found
Jul 13 18:32:47.016345 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:47.016314    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log" limit=1024
Jul 13 18:32:47.071534 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:47.071504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:47.071629 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:47.071541    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:47.071686 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:47.071644    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 18:32:47.071723 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:47.071699    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:48.071685965 +0000 UTC m=+49.202974404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : configmap references non-existent config key: service-ca.crt
Jul 13 18:32:47.071723 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:47.071714    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:48.071708271 +0000 UTC m=+49.202996709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : secret "router-metrics-certs-default" not found
Jul 13 18:32:47.615728 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:47.615689    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps" event={"ID":"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1","Type":"ContainerStarted","Data":"91b3a7493a658a804e465618d85358227bc49e44b5864e34c39e0ab4524cd0d7"}
Jul 13 18:32:47.878083 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:47.877992    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:47.878222 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:47.878150    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 18:32:47.878222 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:47.878218    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls podName:0b35e240-a9b7-4d64-be6b-bb32082a2073 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:49.878202303 +0000 UTC m=+51.009490742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hrjpr" (UID: "0b35e240-a9b7-4d64-be6b-bb32082a2073") : secret "cluster-monitoring-operator-tls" not found
Jul 13 18:32:47.978838 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:47.978801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:32:47.979010 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:47.978847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:47.979010 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:47.978978    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 18:32:47.979095 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:47.978981    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 18:32:47.979095 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:47.979052    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls podName:3351b763-29c9-477b-b8e7-61dfc79ec616 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:49.979032987 +0000 UTC m=+51.110321425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls") pod "cluster-samples-operator-65db858894-dsjk6" (UID: "3351b763-29c9-477b-b8e7-61dfc79ec616") : secret "samples-operator-tls" not found
Jul 13 18:32:47.979095 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:47.979060    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74598d9c75-tkpgp: secret "image-registry-tls" not found
Jul 13 18:32:47.979198 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:47.979115    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls podName:6be9563e-101a-40e5-bff8-17e166e7931e nodeName:}" failed. No retries permitted until 2026-07-13 18:32:49.979096433 +0000 UTC m=+51.110384891 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls") pod "image-registry-74598d9c75-tkpgp" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e") : secret "image-registry-tls" not found
Jul 13 18:32:48.079698 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:48.079640    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:48.079698 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:48.079706    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:48.080067 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:48.079800    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 18:32:48.080067 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:48.079875    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:50.079855564 +0000 UTC m=+51.211144006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : configmap references non-existent config key: service-ca.crt
Jul 13 18:32:48.080067 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:48.079901    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:50.079890143 +0000 UTC m=+51.211178582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : secret "router-metrics-certs-default" not found
Jul 13 18:32:49.894570 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:49.894477    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:49.894970 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:49.894628    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 18:32:49.894970 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:49.894707    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls podName:0b35e240-a9b7-4d64-be6b-bb32082a2073 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:53.89469196 +0000 UTC m=+55.025980398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hrjpr" (UID: "0b35e240-a9b7-4d64-be6b-bb32082a2073") : secret "cluster-monitoring-operator-tls" not found
Jul 13 18:32:49.995451 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:49.995415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:32:49.995451 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:49.995452    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:49.995644 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:49.995550    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 18:32:49.995644 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:49.995556    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 18:32:49.995644 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:49.995561    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74598d9c75-tkpgp: secret "image-registry-tls" not found
Jul 13 18:32:49.995785 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:49.995622    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls podName:3351b763-29c9-477b-b8e7-61dfc79ec616 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:53.995606616 +0000 UTC m=+55.126895054 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls") pod "cluster-samples-operator-65db858894-dsjk6" (UID: "3351b763-29c9-477b-b8e7-61dfc79ec616") : secret "samples-operator-tls" not found
Jul 13 18:32:49.995785 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:49.995703    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls podName:6be9563e-101a-40e5-bff8-17e166e7931e nodeName:}" failed. No retries permitted until 2026-07-13 18:32:53.99568802 +0000 UTC m=+55.126976458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls") pod "image-registry-74598d9c75-tkpgp" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e") : secret "image-registry-tls" not found
Jul 13 18:32:50.096302 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:50.096259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:50.096487 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:50.096437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:50.096487 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:50.096454    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:54.096433049 +0000 UTC m=+55.227721489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : configmap references non-existent config key: service-ca.crt
Jul 13 18:32:50.096607 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:50.096553    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 18:32:50.096684 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:50.096614    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:32:54.096597878 +0000 UTC m=+55.227886330 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : secret "router-metrics-certs-default" not found
Jul 13 18:32:50.624093 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:50.624058    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps" event={"ID":"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1","Type":"ContainerStarted","Data":"5d8f820d12819416d88111dc31be4e5cd81b439a016e9dfd1b490ca70ba3a169"}
Jul 13 18:32:50.644636 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:50.644591    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps" podStartSLOduration=1.751307741 podStartE2EDuration="4.644576131s" totalImagesPullingTime="2.89326839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:32:50.643393949 +0000 UTC m=+51.774682411" watchObservedRunningTime="2026-07-13 18:32:50.644576131 +0000 UTC m=+51.775864569"
Jul 13 18:32:52.112636 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:52.112595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:32:52.113118 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:52.112759    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 18:32:52.113118 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:52.112824    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls podName:60f2abdb-0f87-4d7b-abdd-b9dfef4eb951 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:08.112807363 +0000 UTC m=+69.244095809 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls") pod "dns-default-s8dct" (UID: "60f2abdb-0f87-4d7b-abdd-b9dfef4eb951") : secret "dns-default-metrics-tls" not found
Jul 13 18:32:53.533856 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.533824    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kcsc8"]
Jul 13 18:32:53.557939 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.557913    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.559516 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.559489    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kcsc8"]
Jul 13 18:32:53.560316 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.560297    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 18:32:53.561202 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.561183    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 18:32:53.561312 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.561208    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vfssh\""
Jul 13 18:32:53.561312 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.561265    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 18:32:53.561312 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.561216    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 18:32:53.624250 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.624217    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9cmn\" (UniqueName: \"kubernetes.io/projected/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-api-access-p9cmn\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.624397 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.624260    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf38ff56-4be5-4117-814f-a64b1b5d44ea-crio-socket\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.624397 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.624385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf38ff56-4be5-4117-814f-a64b1b5d44ea-data-volume\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.624539 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.624410    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.624539 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.624427    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.725365 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.725333    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf38ff56-4be5-4117-814f-a64b1b5d44ea-data-volume\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.725549 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.725376    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.725549 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.725396    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.725549 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:53.725482    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 18:32:53.725750 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:53.725489    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 18:32:53.725750 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:53.725570    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm podName:cf38ff56-4be5-4117-814f-a64b1b5d44ea nodeName:}" failed. No retries permitted until 2026-07-13 18:32:54.225548764 +0000 UTC m=+55.356837204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kcsc8" (UID: "cf38ff56-4be5-4117-814f-a64b1b5d44ea") : configmap "kube-rbac-proxy" not found
Jul 13 18:32:53.725750 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:53.725607    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls podName:cf38ff56-4be5-4117-814f-a64b1b5d44ea nodeName:}" failed. No retries permitted until 2026-07-13 18:32:54.225593601 +0000 UTC m=+55.356882039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kcsc8" (UID: "cf38ff56-4be5-4117-814f-a64b1b5d44ea") : secret "insights-runtime-extractor-tls" not found
Jul 13 18:32:53.725750 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.725599    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9cmn\" (UniqueName: \"kubernetes.io/projected/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-api-access-p9cmn\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.725750 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.725644    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf38ff56-4be5-4117-814f-a64b1b5d44ea-crio-socket\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.725977 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.725851    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf38ff56-4be5-4117-814f-a64b1b5d44ea-data-volume\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.725977 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.725902    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf38ff56-4be5-4117-814f-a64b1b5d44ea-crio-socket\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.736960 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.736933    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9cmn\" (UniqueName: \"kubernetes.io/projected/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-api-access-p9cmn\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:53.926586 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:53.926507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:32:53.926757 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:53.926711    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 18:32:53.926821 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:53.926808    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls podName:0b35e240-a9b7-4d64-be6b-bb32082a2073 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:01.926785955 +0000 UTC m=+63.058074407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hrjpr" (UID: "0b35e240-a9b7-4d64-be6b-bb32082a2073") : secret "cluster-monitoring-operator-tls" not found
Jul 13 18:32:54.027279 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:54.027245    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:32:54.027279 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:54.027283    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:32:54.027471 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.027401    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 18:32:54.027513 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.027476    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls podName:3351b763-29c9-477b-b8e7-61dfc79ec616 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:02.027462347 +0000 UTC m=+63.158750785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls") pod "cluster-samples-operator-65db858894-dsjk6" (UID: "3351b763-29c9-477b-b8e7-61dfc79ec616") : secret "samples-operator-tls" not found
Jul 13 18:32:54.027513 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.027404    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 18:32:54.027604 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.027514    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74598d9c75-tkpgp: secret "image-registry-tls" not found
Jul 13 18:32:54.027604 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.027556    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls podName:6be9563e-101a-40e5-bff8-17e166e7931e nodeName:}" failed. No retries permitted until 2026-07-13 18:33:02.027542965 +0000 UTC m=+63.158831406 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls") pod "image-registry-74598d9c75-tkpgp" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e") : secret "image-registry-tls" not found
Jul 13 18:32:54.128277 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:54.128247    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:54.128438 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:54.128285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:32:54.128438 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.128410    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:02.128391818 +0000 UTC m=+63.259680256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : configmap references non-existent config key: service-ca.crt
Jul 13 18:32:54.128438 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.128416    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 18:32:54.128546 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.128482    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:02.128463429 +0000 UTC m=+63.259751873 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : secret "router-metrics-certs-default" not found
Jul 13 18:32:54.229421 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:54.229389    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:54.229594 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:54.229429    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:54.229594 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.229506    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 18:32:54.229594 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.229569    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm podName:cf38ff56-4be5-4117-814f-a64b1b5d44ea nodeName:}" failed. No retries permitted until 2026-07-13 18:32:55.229554055 +0000 UTC m=+56.360842493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kcsc8" (UID: "cf38ff56-4be5-4117-814f-a64b1b5d44ea") : configmap "kube-rbac-proxy" not found
Jul 13 18:32:54.229777 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.229595    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 18:32:54.229777 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:54.229686    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls podName:cf38ff56-4be5-4117-814f-a64b1b5d44ea nodeName:}" failed. No retries permitted until 2026-07-13 18:32:55.22964287 +0000 UTC m=+56.360931309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kcsc8" (UID: "cf38ff56-4be5-4117-814f-a64b1b5d44ea") : secret "insights-runtime-extractor-tls" not found
Jul 13 18:32:55.237828 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:55.237799    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:55.238273 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:55.237837    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:55.238273 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:55.237940    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 18:32:55.238273 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:55.238022    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm podName:cf38ff56-4be5-4117-814f-a64b1b5d44ea nodeName:}" failed. No retries permitted until 2026-07-13 18:32:57.238001149 +0000 UTC m=+58.369289586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kcsc8" (UID: "cf38ff56-4be5-4117-814f-a64b1b5d44ea") : configmap "kube-rbac-proxy" not found
Jul 13 18:32:55.238273 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:55.238032    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 18:32:55.238273 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:55.238080    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls podName:cf38ff56-4be5-4117-814f-a64b1b5d44ea nodeName:}" failed. No retries permitted until 2026-07-13 18:32:57.238063906 +0000 UTC m=+58.369352346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kcsc8" (UID: "cf38ff56-4be5-4117-814f-a64b1b5d44ea") : secret "insights-runtime-extractor-tls" not found
Jul 13 18:32:55.911241 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:55.911213    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-88fdd_91a5e102-1948-414f-bfc4-a468dfe9f3ba/dns-node-resolver/0.log"
Jul 13 18:32:57.110921 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:57.110893    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qs9wn_d153a23e-6499-4764-ad85-666ef8b2dc72/node-ca/0.log"
Jul 13 18:32:57.256238 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:57.256189    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:57.256238 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:57.256242    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:32:57.256452 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:57.256317    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 18:32:57.256452 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:57.256351    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 18:32:57.256452 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:57.256382    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm podName:cf38ff56-4be5-4117-814f-a64b1b5d44ea nodeName:}" failed. No retries permitted until 2026-07-13 18:33:01.256366569 +0000 UTC m=+62.387655007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kcsc8" (UID: "cf38ff56-4be5-4117-814f-a64b1b5d44ea") : configmap "kube-rbac-proxy" not found
Jul 13 18:32:57.256452 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:32:57.256400    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls podName:cf38ff56-4be5-4117-814f-a64b1b5d44ea nodeName:}" failed. No retries permitted until 2026-07-13 18:33:01.25638874 +0000 UTC m=+62.387677178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kcsc8" (UID: "cf38ff56-4be5-4117-814f-a64b1b5d44ea") : secret "insights-runtime-extractor-tls" not found
Jul 13 18:32:57.510584 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:57.510558    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nlnzn_d1729c34-3c2c-4fa7-a94e-f7628fad363e/serve-healthcheck-canary/0.log"
Jul 13 18:32:58.319983 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:32:58.319943    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-cndps_cc90ccc1-03cf-49db-bfd4-5458ec6e63a1/kube-storage-version-migrator-operator/0.log"
Jul 13 18:33:01.288945 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:01.288902    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:33:01.289319 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:01.289055    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:33:01.289319 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:01.289061    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 18:33:01.289319 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:01.289123    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls podName:cf38ff56-4be5-4117-814f-a64b1b5d44ea nodeName:}" failed. No retries permitted until 2026-07-13 18:33:09.289107297 +0000 UTC m=+70.420395736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kcsc8" (UID: "cf38ff56-4be5-4117-814f-a64b1b5d44ea") : secret "insights-runtime-extractor-tls" not found
Jul 13 18:33:01.289319 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:01.289134    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 18:33:01.289319 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:01.289176    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm podName:cf38ff56-4be5-4117-814f-a64b1b5d44ea nodeName:}" failed. No retries permitted until 2026-07-13 18:33:09.289162673 +0000 UTC m=+70.420451111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kcsc8" (UID: "cf38ff56-4be5-4117-814f-a64b1b5d44ea") : configmap "kube-rbac-proxy" not found
Jul 13 18:33:01.995768 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:01.995733    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:33:01.995925 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:01.995881    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 18:33:01.995968 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:01.995949    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls podName:0b35e240-a9b7-4d64-be6b-bb32082a2073 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:17.995934412 +0000 UTC m=+79.127222850 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-hrjpr" (UID: "0b35e240-a9b7-4d64-be6b-bb32082a2073") : secret "cluster-monitoring-operator-tls" not found
Jul 13 18:33:02.096417 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.096374    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:33:02.096417 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.096416    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:33:02.099093 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.099063    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3351b763-29c9-477b-b8e7-61dfc79ec616-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dsjk6\" (UID: \"3351b763-29c9-477b-b8e7-61dfc79ec616\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:33:02.110363 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.110338    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls\") pod \"image-registry-74598d9c75-tkpgp\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") " pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:33:02.155144 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.155113    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gx49k\""
Jul 13 18:33:02.163265 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.163242    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:33:02.189844 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.189815    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nh4mn\""
Jul 13 18:33:02.196600 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.196575    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"
Jul 13 18:33:02.196841 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.196816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:33:02.197014 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.196993    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:33:02.197014 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:02.197008    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle podName:f5c0e7af-b62f-4dfc-810f-39223c968ce9 nodeName:}" failed. No retries permitted until 2026-07-13 18:33:18.196986893 +0000 UTC m=+79.328275332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle") pod "router-default-597d7847fb-55tbz" (UID: "f5c0e7af-b62f-4dfc-810f-39223c968ce9") : configmap references non-existent config key: service-ca.crt
Jul 13 18:33:02.199665 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.199632    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5c0e7af-b62f-4dfc-810f-39223c968ce9-metrics-certs\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:33:02.328636 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.328606    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74598d9c75-tkpgp"]
Jul 13 18:33:02.331925 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:02.331902    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be9563e_101a_40e5_bff8_17e166e7931e.slice/crio-3d9ec7a1b8511df720ad3a2acfe8b972cba79a4b9f47c7545054af167014d19d WatchSource:0}: Error finding container 3d9ec7a1b8511df720ad3a2acfe8b972cba79a4b9f47c7545054af167014d19d: Status 404 returned error can't find the container with id 3d9ec7a1b8511df720ad3a2acfe8b972cba79a4b9f47c7545054af167014d19d
Jul 13 18:33:02.348618 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.348593    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6"]
Jul 13 18:33:02.577180 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.577111    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r6wmn"
Jul 13 18:33:02.655392 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.655356    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp" event={"ID":"6be9563e-101a-40e5-bff8-17e166e7931e","Type":"ContainerStarted","Data":"1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412"}
Jul 13 18:33:02.655561 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.655408    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp" event={"ID":"6be9563e-101a-40e5-bff8-17e166e7931e","Type":"ContainerStarted","Data":"3d9ec7a1b8511df720ad3a2acfe8b972cba79a4b9f47c7545054af167014d19d"}
Jul 13 18:33:02.655561 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.655475    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:33:02.656509 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.656486    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6" event={"ID":"3351b763-29c9-477b-b8e7-61dfc79ec616","Type":"ContainerStarted","Data":"890c5e2636b5e493f757331a27cea3bc1b0523d3fbdec54d879ca0bdd64fae2b"}
Jul 13 18:33:02.739110 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:02.739053    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp" podStartSLOduration=16.739035473 podStartE2EDuration="16.739035473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:02.738316263 +0000 UTC m=+63.869604723" watchObservedRunningTime="2026-07-13 18:33:02.739035473 +0000 UTC m=+63.870323932"
Jul 13 18:33:05.021423 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.021372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:33:05.023869 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.023851    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 18:33:05.034817 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.034793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe11421-9613-4ca6-8c39-c44b3682f360-original-pull-secret\") pod \"global-pull-secret-syncer-ffsc6\" (UID: \"8fe11421-9613-4ca6-8c39-c44b3682f360\") " pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:33:05.121830 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.121794    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:33:05.124170 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.124154    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 18:33:05.132230 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:05.132212    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 13 18:33:05.132280 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:05.132261    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs podName:698c185e-1b6a-43a7-97fb-725ad56c0962 nodeName:}" failed. No retries permitted until 2026-07-13 18:34:09.132247419 +0000 UTC m=+130.263535858 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs") pod "network-metrics-daemon-zvvkh" (UID: "698c185e-1b6a-43a7-97fb-725ad56c0962") : secret "metrics-daemon-secret" not found
Jul 13 18:33:05.156691 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.156666    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffsc6"
Jul 13 18:33:05.222505 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.222475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchnm\" (UniqueName: \"kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm\") pod \"network-check-target-blbvt\" (UID: \"99b3b7e2-f203-4325-945a-214b3a386b4a\") " pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:33:05.225350 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.225325    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 18:33:05.236229 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.236203    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 18:33:05.246863 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.246840    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bchnm\" (UniqueName: \"kubernetes.io/projected/99b3b7e2-f203-4325-945a-214b3a386b4a-kube-api-access-bchnm\") pod \"network-check-target-blbvt\" (UID: \"99b3b7e2-f203-4325-945a-214b3a386b4a\") " pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:33:05.285794 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.285724    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ffsc6"]
Jul 13 18:33:05.288949 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:05.288926    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe11421_9613_4ca6_8c39_c44b3682f360.slice/crio-25c58c1945649101d96dddc22b0b0d589c92c3f3732fe5676ea8e6f8edefe8de WatchSource:0}: Error finding container 25c58c1945649101d96dddc22b0b0d589c92c3f3732fe5676ea8e6f8edefe8de: Status 404 returned error can't find the container with id 25c58c1945649101d96dddc22b0b0d589c92c3f3732fe5676ea8e6f8edefe8de
Jul 13 18:33:05.470879 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.470841    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ql2kw\""
Jul 13 18:33:05.478822 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.478801    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:33:05.601791 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.601761    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-blbvt"]
Jul 13 18:33:05.604952 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:05.604918    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99b3b7e2_f203_4325_945a_214b3a386b4a.slice/crio-3b93c097629bfd22c11b04e856c3e2689a6cc22df77b30ffe7a42adaf1d35726 WatchSource:0}: Error finding container 3b93c097629bfd22c11b04e856c3e2689a6cc22df77b30ffe7a42adaf1d35726: Status 404 returned error can't find the container with id 3b93c097629bfd22c11b04e856c3e2689a6cc22df77b30ffe7a42adaf1d35726
Jul 13 18:33:05.666411 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.666378    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6" event={"ID":"3351b763-29c9-477b-b8e7-61dfc79ec616","Type":"ContainerStarted","Data":"4a01758f3809e181f9a58c123c788f8c70ffe293f1551556886e89675e1297ad"}
Jul 13 18:33:05.666572 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.666435    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6" event={"ID":"3351b763-29c9-477b-b8e7-61dfc79ec616","Type":"ContainerStarted","Data":"5f0d00afe75b50421d0457fa6fb43a17e43adf8ff5a2cfc3c04e7cc535f1dd97"}
Jul 13 18:33:05.667559 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.667530    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-blbvt" event={"ID":"99b3b7e2-f203-4325-945a-214b3a386b4a","Type":"ContainerStarted","Data":"3b93c097629bfd22c11b04e856c3e2689a6cc22df77b30ffe7a42adaf1d35726"}
Jul 13 18:33:05.668480 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.668456    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ffsc6" event={"ID":"8fe11421-9613-4ca6-8c39-c44b3682f360","Type":"ContainerStarted","Data":"25c58c1945649101d96dddc22b0b0d589c92c3f3732fe5676ea8e6f8edefe8de"}
Jul 13 18:33:05.685403 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:05.685353    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dsjk6" podStartSLOduration=17.372829644 podStartE2EDuration="19.685337781s" totalImagesPullingTime="2.312508137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:05.684812998 +0000 UTC m=+66.816101457" watchObservedRunningTime="2026-07-13 18:33:05.685337781 +0000 UTC m=+66.816626240"
Jul 13 18:33:08.150060 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:08.150021    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:33:08.152850 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:08.152825    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60f2abdb-0f87-4d7b-abdd-b9dfef4eb951-metrics-tls\") pod \"dns-default-s8dct\" (UID: \"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951\") " pod="openshift-dns/dns-default-s8dct"
Jul 13 18:33:08.409949 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:08.409874    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tnbnw\""
Jul 13 18:33:08.417470 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:08.417445    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s8dct"
Jul 13 18:33:08.766936 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:08.766909    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s8dct"]
Jul 13 18:33:08.770085 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:08.770062    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f2abdb_0f87_4d7b_abdd_b9dfef4eb951.slice/crio-a3aa7335109af762008e737465636a9a50c802c0a87cd011938c6f5821e75326 WatchSource:0}: Error finding container a3aa7335109af762008e737465636a9a50c802c0a87cd011938c6f5821e75326: Status 404 returned error can't find the container with id a3aa7335109af762008e737465636a9a50c802c0a87cd011938c6f5821e75326
Jul 13 18:33:09.360157 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.360060    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:33:09.360157 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.360121    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:33:09.360792 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.360760    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf38ff56-4be5-4117-814f-a64b1b5d44ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:33:09.363105 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.363078    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf38ff56-4be5-4117-814f-a64b1b5d44ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcsc8\" (UID: \"cf38ff56-4be5-4117-814f-a64b1b5d44ea\") " pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:33:09.470664 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.470626    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vfssh\""
Jul 13 18:33:09.478767 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.478735    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kcsc8"
Jul 13 18:33:09.660217 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.660141    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kcsc8"]
Jul 13 18:33:09.667699 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:09.664105    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf38ff56_4be5_4117_814f_a64b1b5d44ea.slice/crio-eb95a6b6ccb776b15ae893e212c775bf5c7e0c1477c29402c698d988cd3c1160 WatchSource:0}: Error finding container eb95a6b6ccb776b15ae893e212c775bf5c7e0c1477c29402c698d988cd3c1160: Status 404 returned error can't find the container with id eb95a6b6ccb776b15ae893e212c775bf5c7e0c1477c29402c698d988cd3c1160
Jul 13 18:33:09.692236 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.692201    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-blbvt" event={"ID":"99b3b7e2-f203-4325-945a-214b3a386b4a","Type":"ContainerStarted","Data":"23839b8d7bf308e3bbe7565b7397e83ec0e8f7f211722439b097f8c5fc0d1a1c"}
Jul 13 18:33:09.692454 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.692429    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:33:09.693809 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.693782    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kcsc8" event={"ID":"cf38ff56-4be5-4117-814f-a64b1b5d44ea","Type":"ContainerStarted","Data":"eb95a6b6ccb776b15ae893e212c775bf5c7e0c1477c29402c698d988cd3c1160"}
Jul 13 18:33:09.695070 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.695041    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s8dct" event={"ID":"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951","Type":"ContainerStarted","Data":"a3aa7335109af762008e737465636a9a50c802c0a87cd011938c6f5821e75326"}
Jul 13 18:33:09.709853 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:09.709798    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-blbvt" podStartSLOduration=66.306661513 podStartE2EDuration="1m9.709783568s" totalImagesPullingTime="3.403122055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:09.708638594 +0000 UTC m=+70.839927056" watchObservedRunningTime="2026-07-13 18:33:09.709783568 +0000 UTC m=+70.841072005"
Jul 13 18:33:11.705898 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:11.705859    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s8dct" event={"ID":"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951","Type":"ContainerStarted","Data":"10de21f64f39d8e5873530021ace3a771253e74ff3da49e249acc8a2610a20a8"}
Jul 13 18:33:11.706316 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:11.705924    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s8dct" event={"ID":"60f2abdb-0f87-4d7b-abdd-b9dfef4eb951","Type":"ContainerStarted","Data":"f99e5fd041f4913d8995da7c7e5ef861e0b204b5875b5a1b992609c77bbcf0fd"}
Jul 13 18:33:11.706316 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:11.706024    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-s8dct"
Jul 13 18:33:11.740413 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:11.740363    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-s8dct" podStartSLOduration=33.403950552 podStartE2EDuration="35.740342742s" totalImagesPullingTime="2.33639219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:11.739991978 +0000 UTC m=+72.871280440" watchObservedRunningTime="2026-07-13 18:33:11.740342742 +0000 UTC m=+72.871631213"
Jul 13 18:33:13.715490 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:13.715446    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ffsc6" event={"ID":"8fe11421-9613-4ca6-8c39-c44b3682f360","Type":"ContainerStarted","Data":"59dc233c22efd5a117f10ee06057ac977b7d06dd85bd3600231747fc6c9a5e52"}
Jul 13 18:33:13.739235 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:13.739175    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ffsc6" podStartSLOduration=65.960131411 podStartE2EDuration="1m13.73915383s" totalImagesPullingTime="7.779022419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:13.738262899 +0000 UTC m=+74.869551360" watchObservedRunningTime="2026-07-13 18:33:13.73915383 +0000 UTC m=+74.870442290"
Jul 13 18:33:16.234639 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.234015    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74598d9c75-tkpgp"]
Jul 13 18:33:16.235418 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.235377    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"]
Jul 13 18:33:16.243465 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.243441    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.246224 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.246045    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 13 18:33:16.246224 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.246123    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 13 18:33:16.246483 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.246463    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 13 18:33:16.246602 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.246470    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 13 18:33:16.259747 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.259724    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"]
Jul 13 18:33:16.320756 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.320727    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a25eb772-f6c9-46b9-8fdb-31145ceb570c-tmp\") pod \"klusterlet-addon-workmgr-cb98fcc9-crqsr\" (UID: \"a25eb772-f6c9-46b9-8fdb-31145ceb570c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.320933 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.320763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a25eb772-f6c9-46b9-8fdb-31145ceb570c-klusterlet-config\") pod \"klusterlet-addon-workmgr-cb98fcc9-crqsr\" (UID: \"a25eb772-f6c9-46b9-8fdb-31145ceb570c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.320933 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.320890    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q7lk\" (UniqueName: \"kubernetes.io/projected/a25eb772-f6c9-46b9-8fdb-31145ceb570c-kube-api-access-6q7lk\") pod \"klusterlet-addon-workmgr-cb98fcc9-crqsr\" (UID: \"a25eb772-f6c9-46b9-8fdb-31145ceb570c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.352904 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.352867    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bd58b54dc-59p46"]
Jul 13 18:33:16.356098 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.356075    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.400049 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.400020    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bd58b54dc-59p46"]
Jul 13 18:33:16.422142 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422110    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a25eb772-f6c9-46b9-8fdb-31145ceb570c-tmp\") pod \"klusterlet-addon-workmgr-cb98fcc9-crqsr\" (UID: \"a25eb772-f6c9-46b9-8fdb-31145ceb570c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.422284 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422146    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a25eb772-f6c9-46b9-8fdb-31145ceb570c-klusterlet-config\") pod \"klusterlet-addon-workmgr-cb98fcc9-crqsr\" (UID: \"a25eb772-f6c9-46b9-8fdb-31145ceb570c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.422284 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422176    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-trusted-ca\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.422284 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422219    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-registry-tls\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.422284 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422262    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-bound-sa-token\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.422433 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422329    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-registry-certificates\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.422433 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422389    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q7lk\" (UniqueName: \"kubernetes.io/projected/a25eb772-f6c9-46b9-8fdb-31145ceb570c-kube-api-access-6q7lk\") pod \"klusterlet-addon-workmgr-cb98fcc9-crqsr\" (UID: \"a25eb772-f6c9-46b9-8fdb-31145ceb570c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.422433 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-installation-pull-secrets\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.422569 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422434    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-image-registry-private-configuration\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.422569 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422464    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a25eb772-f6c9-46b9-8fdb-31145ceb570c-tmp\") pod \"klusterlet-addon-workmgr-cb98fcc9-crqsr\" (UID: \"a25eb772-f6c9-46b9-8fdb-31145ceb570c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.422569 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbfxl\" (UniqueName: \"kubernetes.io/projected/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-kube-api-access-bbfxl\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.422569 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.422527    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-ca-trust-extracted\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.424703 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.424682    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a25eb772-f6c9-46b9-8fdb-31145ceb570c-klusterlet-config\") pod \"klusterlet-addon-workmgr-cb98fcc9-crqsr\" (UID: \"a25eb772-f6c9-46b9-8fdb-31145ceb570c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.439168 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.439143    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q7lk\" (UniqueName: \"kubernetes.io/projected/a25eb772-f6c9-46b9-8fdb-31145ceb570c-kube-api-access-6q7lk\") pod \"klusterlet-addon-workmgr-cb98fcc9-crqsr\" (UID: \"a25eb772-f6c9-46b9-8fdb-31145ceb570c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.523862 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.523836    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-installation-pull-secrets\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.523983 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.523872    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-image-registry-private-configuration\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.523983 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.523894    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbfxl\" (UniqueName: \"kubernetes.io/projected/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-kube-api-access-bbfxl\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.523983 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.523924    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-ca-trust-extracted\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.523983 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.523977    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-trusted-ca\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.524191 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.524012    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-registry-tls\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.524191 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.524054    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-bound-sa-token\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.524191 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.524086    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-registry-certificates\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.524409 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.524389    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-ca-trust-extracted\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.525533 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.525507    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-registry-certificates\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.525698 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.525645    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-trusted-ca\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.526765 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.526746    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-installation-pull-secrets\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.526866 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.526843    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-image-registry-private-configuration\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.527054 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.527034    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-registry-tls\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.541355 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.541336    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbfxl\" (UniqueName: \"kubernetes.io/projected/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-kube-api-access-bbfxl\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.541443 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.541367    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d47f9940-79d2-41a7-bc33-bbb53cc5b89b-bound-sa-token\") pod \"image-registry-bd58b54dc-59p46\" (UID: \"d47f9940-79d2-41a7-bc33-bbb53cc5b89b\") " pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.556315 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.556298    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:16.668219 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.668191    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:16.705111 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.705086    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"]
Jul 13 18:33:16.708352 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:16.708315    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25eb772_f6c9_46b9_8fdb_31145ceb570c.slice/crio-382c8841bd70b2340f1b4e65e94a28813129c755fa804ba9d555207aaf090186 WatchSource:0}: Error finding container 382c8841bd70b2340f1b4e65e94a28813129c755fa804ba9d555207aaf090186: Status 404 returned error can't find the container with id 382c8841bd70b2340f1b4e65e94a28813129c755fa804ba9d555207aaf090186
Jul 13 18:33:16.728241 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.728212    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kcsc8" event={"ID":"cf38ff56-4be5-4117-814f-a64b1b5d44ea","Type":"ContainerStarted","Data":"b14e3c16616d2c538ad532d61201c4e191394a08d8182dbb829a8351b21b2ef8"}
Jul 13 18:33:16.729481 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.729460    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr" event={"ID":"a25eb772-f6c9-46b9-8fdb-31145ceb570c","Type":"ContainerStarted","Data":"382c8841bd70b2340f1b4e65e94a28813129c755fa804ba9d555207aaf090186"}
Jul 13 18:33:16.815638 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:16.815603    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bd58b54dc-59p46"]
Jul 13 18:33:16.822681 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:16.822628    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47f9940_79d2_41a7_bc33_bbb53cc5b89b.slice/crio-71dfe27af4399acc29c45a1acb5b7a07f582e805bfb03f58988f04ffc17b5416 WatchSource:0}: Error finding container 71dfe27af4399acc29c45a1acb5b7a07f582e805bfb03f58988f04ffc17b5416: Status 404 returned error can't find the container with id 71dfe27af4399acc29c45a1acb5b7a07f582e805bfb03f58988f04ffc17b5416
Jul 13 18:33:17.737005 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:17.736969    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kcsc8" event={"ID":"cf38ff56-4be5-4117-814f-a64b1b5d44ea","Type":"ContainerStarted","Data":"9867a960716a84042cecdfbfc19bc58cd381032432b4b885b6a9add0d64677aa"}
Jul 13 18:33:17.738667 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:17.738618    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bd58b54dc-59p46" event={"ID":"d47f9940-79d2-41a7-bc33-bbb53cc5b89b","Type":"ContainerStarted","Data":"fa39c79e2412499ee438bbb7be9ad330375bee44b6ea631263010aaefe095c3b"}
Jul 13 18:33:17.738801 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:17.738691    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bd58b54dc-59p46" event={"ID":"d47f9940-79d2-41a7-bc33-bbb53cc5b89b","Type":"ContainerStarted","Data":"71dfe27af4399acc29c45a1acb5b7a07f582e805bfb03f58988f04ffc17b5416"}
Jul 13 18:33:17.738801 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:17.738760    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:18.035876 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:18.035778    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:33:18.041809 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:18.041759    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b35e240-a9b7-4d64-be6b-bb32082a2073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-hrjpr\" (UID: \"0b35e240-a9b7-4d64-be6b-bb32082a2073\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:33:18.226355 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:18.226323    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tgsln\""
Jul 13 18:33:18.234421 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:18.234394    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"
Jul 13 18:33:18.238090 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:18.238057    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:33:18.238914 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:18.238896    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5c0e7af-b62f-4dfc-810f-39223c968ce9-service-ca-bundle\") pod \"router-default-597d7847fb-55tbz\" (UID: \"f5c0e7af-b62f-4dfc-810f-39223c968ce9\") " pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:33:18.412414 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:18.412334    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gghh4\""
Jul 13 18:33:18.420360 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:18.420131    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:33:19.012909 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:19.012847    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bd58b54dc-59p46" podStartSLOduration=3.012821959 podStartE2EDuration="3.012821959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:17.779292755 +0000 UTC m=+78.910581215" watchObservedRunningTime="2026-07-13 18:33:19.012821959 +0000 UTC m=+80.144110420"
Jul 13 18:33:19.013321 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:19.013239    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-597d7847fb-55tbz"]
Jul 13 18:33:19.017948 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:19.017764    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5c0e7af_b62f_4dfc_810f_39223c968ce9.slice/crio-fa953a0dbb146faee49e3be6f86c3a0d0f26ac58d71ba0e59fbd02a4928e01dc WatchSource:0}: Error finding container fa953a0dbb146faee49e3be6f86c3a0d0f26ac58d71ba0e59fbd02a4928e01dc: Status 404 returned error can't find the container with id fa953a0dbb146faee49e3be6f86c3a0d0f26ac58d71ba0e59fbd02a4928e01dc
Jul 13 18:33:19.023073 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:19.023012    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr"]
Jul 13 18:33:19.028771 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:19.028738    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b35e240_a9b7_4d64_be6b_bb32082a2073.slice/crio-8def0bad4bf4e6275e8832865c702f7b0130005b3268a935b944dd2e55574e5c WatchSource:0}: Error finding container 8def0bad4bf4e6275e8832865c702f7b0130005b3268a935b944dd2e55574e5c: Status 404 returned error can't find the container with id 8def0bad4bf4e6275e8832865c702f7b0130005b3268a935b944dd2e55574e5c
Jul 13 18:33:19.748588 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:19.748441    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr" event={"ID":"0b35e240-a9b7-4d64-be6b-bb32082a2073","Type":"ContainerStarted","Data":"8def0bad4bf4e6275e8832865c702f7b0130005b3268a935b944dd2e55574e5c"}
Jul 13 18:33:19.750140 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:19.750113    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-597d7847fb-55tbz" event={"ID":"f5c0e7af-b62f-4dfc-810f-39223c968ce9","Type":"ContainerStarted","Data":"b80f74c24967383a3ad5a92a98d941731808ced193dea96fd462d6b01b7df1ab"}
Jul 13 18:33:19.750272 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:19.750166    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-597d7847fb-55tbz" event={"ID":"f5c0e7af-b62f-4dfc-810f-39223c968ce9","Type":"ContainerStarted","Data":"fa953a0dbb146faee49e3be6f86c3a0d0f26ac58d71ba0e59fbd02a4928e01dc"}
Jul 13 18:33:19.775334 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:19.775270    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-597d7847fb-55tbz" podStartSLOduration=33.775251203 podStartE2EDuration="33.775251203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:19.773592244 +0000 UTC m=+80.904880706" watchObservedRunningTime="2026-07-13 18:33:19.775251203 +0000 UTC m=+80.906539666"
Jul 13 18:33:20.421007 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:20.420972    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:33:20.423889 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:20.423864    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:33:20.754097 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:20.754063    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:33:20.755896 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:20.755873    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-597d7847fb-55tbz"
Jul 13 18:33:21.711341 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:21.711262    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-s8dct"
Jul 13 18:33:21.760387 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:21.760125    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr" event={"ID":"a25eb772-f6c9-46b9-8fdb-31145ceb570c","Type":"ContainerStarted","Data":"6b1feeafdd33e49b04e5e248c158c25c0a28990f7fc9bd4f1de3b87290e64f81"}
Jul 13 18:33:21.760387 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:21.760192    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:21.762747 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:21.762643    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kcsc8" event={"ID":"cf38ff56-4be5-4117-814f-a64b1b5d44ea","Type":"ContainerStarted","Data":"b2d88f8738ea628558033aeef305f5fd356c0a7d2cbc4318023bbf648f16466b"}
Jul 13 18:33:21.763466 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:21.763441    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr"
Jul 13 18:33:21.809238 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:21.809177    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cb98fcc9-crqsr" podStartSLOduration=1.06168297 podStartE2EDuration="5.809158061s" totalImagesPullingTime="4.747475091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:21.808904745 +0000 UTC m=+82.940193213" watchObservedRunningTime="2026-07-13 18:33:21.809158061 +0000 UTC m=+82.940446522"
Jul 13 18:33:21.848909 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:21.848852    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kcsc8" podStartSLOduration=17.206523904 podStartE2EDuration="28.848834241s" totalImagesPullingTime="11.642310337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:32:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:21.848399421 +0000 UTC m=+82.979687882" watchObservedRunningTime="2026-07-13 18:33:21.848834241 +0000 UTC m=+82.980122701"
Jul 13 18:33:23.771043 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:23.770992    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr" event={"ID":"0b35e240-a9b7-4d64-be6b-bb32082a2073","Type":"ContainerStarted","Data":"a9aae505a1adb23c6be60b22d8471fdceae56e460e3cb3b080115c80d5c16497"}
Jul 13 18:33:23.791128 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:23.791075    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-hrjpr" podStartSLOduration=33.733355297 podStartE2EDuration="37.791059726s" totalImagesPullingTime="4.057704429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:23.79060568 +0000 UTC m=+84.921894151" watchObservedRunningTime="2026-07-13 18:33:23.791059726 +0000 UTC m=+84.922348185"
Jul 13 18:33:26.240299 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:26.240266    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:33:33.723588 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.723558    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6vctn"]
Jul 13 18:33:33.728222 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.728202    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.732783 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:33.732758    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"node-exporter-accelerators-collector-config\" is forbidden: User \"system:node:ip-10-0-139-206.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-139-206.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap"
Jul 13 18:33:33.733641 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:33.733617    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-139-206.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-139-206.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret"
Jul 13 18:33:33.734561 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.734542    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jxvkr\""
Jul 13 18:33:33.735718 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.735700    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 18:33:33.735799 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.735701    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 18:33:33.883277 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.883243    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-tls\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.883415 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.883285    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7wrv\" (UniqueName: \"kubernetes.io/projected/feb9153f-0b3c-4777-9781-9132d572cf39-kube-api-access-z7wrv\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.883415 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.883304    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/feb9153f-0b3c-4777-9781-9132d572cf39-sys\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.883501 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.883406    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-textfile\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.883501 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.883456    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-accelerators-collector-config\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.883558 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.883497    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/feb9153f-0b3c-4777-9781-9132d572cf39-metrics-client-ca\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.883558 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.883536    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/feb9153f-0b3c-4777-9781-9132d572cf39-root\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.883620 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.883577    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-wtmp\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.883620 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.883594    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.984794 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.984755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-accelerators-collector-config\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.984982 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.984800    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/feb9153f-0b3c-4777-9781-9132d572cf39-metrics-client-ca\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.984982 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.984830    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/feb9153f-0b3c-4777-9781-9132d572cf39-root\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.984982 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.984866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-wtmp\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.984982 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.984887    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.984982 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.984923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-tls\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.984982 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.984933    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/feb9153f-0b3c-4777-9781-9132d572cf39-root\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.984982 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.984963    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7wrv\" (UniqueName: \"kubernetes.io/projected/feb9153f-0b3c-4777-9781-9132d572cf39-kube-api-access-z7wrv\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.985263 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.984986    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/feb9153f-0b3c-4777-9781-9132d572cf39-sys\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.985263 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.985035    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-textfile\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.985263 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.985037    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-wtmp\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.985263 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.985095    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/feb9153f-0b3c-4777-9781-9132d572cf39-sys\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.985395 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.985327    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-textfile\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.985452 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.985425    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/feb9153f-0b3c-4777-9781-9132d572cf39-metrics-client-ca\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:33.987422 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:33.987399    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-tls\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:34.001433 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:34.001411    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7wrv\" (UniqueName: \"kubernetes.io/projected/feb9153f-0b3c-4777-9781-9132d572cf39-kube-api-access-z7wrv\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:34.579399 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:34.579371    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 18:33:34.586018 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:34.585985    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-accelerators-collector-config\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:34.929551 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:34.929358    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 18:33:34.938307 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:34.938272    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/feb9153f-0b3c-4777-9781-9132d572cf39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6vctn\" (UID: \"feb9153f-0b3c-4777-9781-9132d572cf39\") " pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:35.236879 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:35.236842    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6vctn"
Jul 13 18:33:35.248031 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:35.247997    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeb9153f_0b3c_4777_9781_9132d572cf39.slice/crio-5df6ef5291c0170e13fe6ea6a3aa34e2e13fc9e08ea800746a82e41c13e47d26 WatchSource:0}: Error finding container 5df6ef5291c0170e13fe6ea6a3aa34e2e13fc9e08ea800746a82e41c13e47d26: Status 404 returned error can't find the container with id 5df6ef5291c0170e13fe6ea6a3aa34e2e13fc9e08ea800746a82e41c13e47d26
Jul 13 18:33:35.812632 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:35.812593    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6vctn" event={"ID":"feb9153f-0b3c-4777-9781-9132d572cf39","Type":"ContainerStarted","Data":"5df6ef5291c0170e13fe6ea6a3aa34e2e13fc9e08ea800746a82e41c13e47d26"}
Jul 13 18:33:36.819757 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:36.819317    2569 generic.go:356] "Generic (PLEG): container finished" podID="feb9153f-0b3c-4777-9781-9132d572cf39" containerID="bcde81378067442d7cec203c7d7404f05221328272b5b65f99dfab7e5439bd5f" exitCode=0
Jul 13 18:33:36.819757 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:36.819392    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6vctn" event={"ID":"feb9153f-0b3c-4777-9781-9132d572cf39","Type":"ContainerDied","Data":"bcde81378067442d7cec203c7d7404f05221328272b5b65f99dfab7e5439bd5f"}
Jul 13 18:33:37.824914 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:37.824878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6vctn" event={"ID":"feb9153f-0b3c-4777-9781-9132d572cf39","Type":"ContainerStarted","Data":"584f57a0d7b6edccd510ee730858cc54d7802c49bb6a919be05c0fca05ff19bc"}
Jul 13 18:33:37.825373 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:37.824938    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6vctn" event={"ID":"feb9153f-0b3c-4777-9781-9132d572cf39","Type":"ContainerStarted","Data":"74805f460da94db9be06943deada78adcb3e836cc1b17a90e1509205e5d6a7e2"}
Jul 13 18:33:37.863014 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:37.862961    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6vctn" podStartSLOduration=3.887485757 podStartE2EDuration="4.862945876s" totalImagesPullingTime="902.146341ms" totalInitContainerRuntime="73.313778ms" isStatefulPod=true podCreationTimestamp="2026-07-13 18:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:37.86226643 +0000 UTC m=+98.993554890" watchObservedRunningTime="2026-07-13 18:33:37.862945876 +0000 UTC m=+98.994234367"
Jul 13 18:33:38.748209 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:38.748183    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bd58b54dc-59p46"
Jul 13 18:33:40.703975 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:40.703944    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-blbvt"
Jul 13 18:33:41.260345 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.260290    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp" podUID="6be9563e-101a-40e5-bff8-17e166e7931e" containerName="registry" containerID="cri-o://1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412" gracePeriod=30
Jul 13 18:33:41.505100 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.505075    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:33:41.543512 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.543439    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-trusted-ca\") pod \"6be9563e-101a-40e5-bff8-17e166e7931e\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") "
Jul 13 18:33:41.543512 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.543505    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-image-registry-private-configuration\") pod \"6be9563e-101a-40e5-bff8-17e166e7931e\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") "
Jul 13 18:33:41.543757 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.543603    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-bound-sa-token\") pod \"6be9563e-101a-40e5-bff8-17e166e7931e\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") "
Jul 13 18:33:41.543757 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.543634    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-registry-certificates\") pod \"6be9563e-101a-40e5-bff8-17e166e7931e\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") "
Jul 13 18:33:41.543757 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.543674    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls\") pod \"6be9563e-101a-40e5-bff8-17e166e7931e\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") "
Jul 13 18:33:41.543757 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.543709    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-installation-pull-secrets\") pod \"6be9563e-101a-40e5-bff8-17e166e7931e\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") "
Jul 13 18:33:41.543757 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.543739    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqsxs\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-kube-api-access-bqsxs\") pod \"6be9563e-101a-40e5-bff8-17e166e7931e\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") "
Jul 13 18:33:41.544008 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.543766    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6be9563e-101a-40e5-bff8-17e166e7931e-ca-trust-extracted\") pod \"6be9563e-101a-40e5-bff8-17e166e7931e\" (UID: \"6be9563e-101a-40e5-bff8-17e166e7931e\") "
Jul 13 18:33:41.544296 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.544122    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6be9563e-101a-40e5-bff8-17e166e7931e" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:33:41.544610 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.544575    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6be9563e-101a-40e5-bff8-17e166e7931e" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:33:41.546893 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.546864    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6be9563e-101a-40e5-bff8-17e166e7931e" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:33:41.547013 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.546906    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6be9563e-101a-40e5-bff8-17e166e7931e" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:33:41.547013 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.546959    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-kube-api-access-bqsxs" (OuterVolumeSpecName: "kube-api-access-bqsxs") pod "6be9563e-101a-40e5-bff8-17e166e7931e" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e"). InnerVolumeSpecName "kube-api-access-bqsxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:33:41.547451 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.547432    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6be9563e-101a-40e5-bff8-17e166e7931e" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:33:41.547855 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.547835    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6be9563e-101a-40e5-bff8-17e166e7931e" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:33:41.553619 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.553599    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be9563e-101a-40e5-bff8-17e166e7931e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6be9563e-101a-40e5-bff8-17e166e7931e" (UID: "6be9563e-101a-40e5-bff8-17e166e7931e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:33:41.645335 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.645304    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-bound-sa-token\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:33:41.645335 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.645331    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-registry-certificates\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:33:41.645335 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.645342    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-registry-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:33:41.645542 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.645353    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-installation-pull-secrets\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:33:41.645542 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.645361    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqsxs\" (UniqueName: \"kubernetes.io/projected/6be9563e-101a-40e5-bff8-17e166e7931e-kube-api-access-bqsxs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:33:41.645542 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.645369    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6be9563e-101a-40e5-bff8-17e166e7931e-ca-trust-extracted\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:33:41.645542 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.645377    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6be9563e-101a-40e5-bff8-17e166e7931e-trusted-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:33:41.645542 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.645385    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6be9563e-101a-40e5-bff8-17e166e7931e-image-registry-private-configuration\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:33:41.840403 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.840318    2569 generic.go:356] "Generic (PLEG): container finished" podID="6be9563e-101a-40e5-bff8-17e166e7931e" containerID="1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412" exitCode=0
Jul 13 18:33:41.840403 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.840363    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp"
Jul 13 18:33:41.840868 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.840410    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp" event={"ID":"6be9563e-101a-40e5-bff8-17e166e7931e","Type":"ContainerDied","Data":"1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412"}
Jul 13 18:33:41.840868 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.840460    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74598d9c75-tkpgp" event={"ID":"6be9563e-101a-40e5-bff8-17e166e7931e","Type":"ContainerDied","Data":"3d9ec7a1b8511df720ad3a2acfe8b972cba79a4b9f47c7545054af167014d19d"}
Jul 13 18:33:41.840868 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.840486    2569 scope.go:117] "RemoveContainer" containerID="1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412"
Jul 13 18:33:41.851568 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.851547    2569 scope.go:117] "RemoveContainer" containerID="1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412"
Jul 13 18:33:41.851857 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:33:41.851836    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412\": container with ID starting with 1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412 not found: ID does not exist" containerID="1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412"
Jul 13 18:33:41.851923 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.851868    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412"} err="failed to get container status \"1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412\": rpc error: code = NotFound desc = could not find container \"1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412\": container with ID starting with 1682ab254929f079fb4d505b545baebe69fec2550e9bbb1c03d23ad6c38ec412 not found: ID does not exist"
Jul 13 18:33:41.864047 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.864022    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74598d9c75-tkpgp"]
Jul 13 18:33:41.868404 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:41.868383    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74598d9c75-tkpgp"]
Jul 13 18:33:43.348520 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:43.348484    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6be9563e-101a-40e5-bff8-17e166e7931e" path="/var/lib/kubelet/pods/6be9563e-101a-40e5-bff8-17e166e7931e/volumes"
Jul 13 18:33:44.188085 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.188052    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fc545f97-w9tsl"]
Jul 13 18:33:44.188314 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.188302    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be9563e-101a-40e5-bff8-17e166e7931e" containerName="registry"
Jul 13 18:33:44.188358 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.188315    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be9563e-101a-40e5-bff8-17e166e7931e" containerName="registry"
Jul 13 18:33:44.188391 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.188358    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be9563e-101a-40e5-bff8-17e166e7931e" containerName="registry"
Jul 13 18:33:44.192907 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.192883    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.195708 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.195687    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 13 18:33:44.195818 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.195707    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 13 18:33:44.195818 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.195687    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 13 18:33:44.195818 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.195760    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 13 18:33:44.196911 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.196890    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 13 18:33:44.196911 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.196901    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-76j4m\""
Jul 13 18:33:44.197065 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.196892    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 13 18:33:44.197065 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.196946    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 13 18:33:44.202204 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.202187    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fc545f97-w9tsl"]
Jul 13 18:33:44.267941 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.267907    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-config\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.267941 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.267942    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-oauth-serving-cert\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.268159 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.268008    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-serving-cert\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.268159 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.268032    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-oauth-config\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.268159 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.268049    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-service-ca\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.268159 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.268074    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmhd7\" (UniqueName: \"kubernetes.io/projected/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-kube-api-access-kmhd7\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.368531 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.368496    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-serving-cert\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.369023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.368540    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-oauth-config\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.369023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.368566    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-service-ca\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.369023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.368590    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmhd7\" (UniqueName: \"kubernetes.io/projected/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-kube-api-access-kmhd7\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.369023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.368626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-config\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.369023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.368677    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-oauth-serving-cert\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.369431 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.369407    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-oauth-serving-cert\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.369515 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.369444    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-config\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.369552 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.369534    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-service-ca\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.371747 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.371726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-oauth-config\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.371843 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.371806    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-serving-cert\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.377549 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.377531    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmhd7\" (UniqueName: \"kubernetes.io/projected/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-kube-api-access-kmhd7\") pod \"console-7fc545f97-w9tsl\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") " pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.501640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.501603    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:44.641325 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.641293    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fc545f97-w9tsl"]
Jul 13 18:33:44.644588 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:44.644555    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b8fe07_f8c6_4ed3_bc43_f1e57b5679be.slice/crio-3740dd117b6ae82c3c6e2d5f8fc50be05b2b6ef16383bb0dd5a8feb88ac3bb3f WatchSource:0}: Error finding container 3740dd117b6ae82c3c6e2d5f8fc50be05b2b6ef16383bb0dd5a8feb88ac3bb3f: Status 404 returned error can't find the container with id 3740dd117b6ae82c3c6e2d5f8fc50be05b2b6ef16383bb0dd5a8feb88ac3bb3f
Jul 13 18:33:44.852542 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:44.852454    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc545f97-w9tsl" event={"ID":"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be","Type":"ContainerStarted","Data":"3740dd117b6ae82c3c6e2d5f8fc50be05b2b6ef16383bb0dd5a8feb88ac3bb3f"}
Jul 13 18:33:48.869025 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:48.868939    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc545f97-w9tsl" event={"ID":"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be","Type":"ContainerStarted","Data":"22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04"}
Jul 13 18:33:48.891526 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:48.891478    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fc545f97-w9tsl" podStartSLOduration=0.980883177 podStartE2EDuration="4.891462856s" totalImagesPullingTime="3.910579679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:48.891433019 +0000 UTC m=+110.022721480" watchObservedRunningTime="2026-07-13 18:33:48.891462856 +0000 UTC m=+110.022751307"
Jul 13 18:33:54.501921 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:54.501875    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:54.502348 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:54.501943    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:54.506574 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:54.506553    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:54.893115 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:54.893041    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:33:55.559238 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.559207    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cc4ffb449-cd8vx"]
Jul 13 18:33:55.567492 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.567473    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.574182 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.574159    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc4ffb449-cd8vx"]
Jul 13 18:33:55.575327 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.575305    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 13 18:33:55.659596 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.659563    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-oauth-config\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.659596 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.659597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzvbw\" (UniqueName: \"kubernetes.io/projected/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-kube-api-access-nzvbw\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.659849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.659676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-config\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.659849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.659704    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-trusted-ca-bundle\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.659849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.659728    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-serving-cert\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.659849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.659746    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-oauth-serving-cert\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.659849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.659761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-service-ca\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.761125 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.761085    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-oauth-config\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.761290 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.761128    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzvbw\" (UniqueName: \"kubernetes.io/projected/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-kube-api-access-nzvbw\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.761290 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.761171    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-config\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.761290 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.761193    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-trusted-ca-bundle\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.761290 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.761218    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-serving-cert\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.761290 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.761245    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-oauth-serving-cert\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.761526 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.761307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-service-ca\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.761919 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.761889    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-config\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.762034 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.761988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-service-ca\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.762201 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.762181    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-oauth-serving-cert\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.762336 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.762316    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-trusted-ca-bundle\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.763786 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.763765    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-oauth-config\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.763914 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.763898    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-serving-cert\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.770097 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.770079    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzvbw\" (UniqueName: \"kubernetes.io/projected/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-kube-api-access-nzvbw\") pod \"console-7cc4ffb449-cd8vx\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") " pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:55.877764 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:55.877689    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:33:56.012763 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:56.012731    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc4ffb449-cd8vx"]
Jul 13 18:33:56.016198 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:33:56.016163    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34aa3fe_5cdc_4e1c_8d6a_0129f3c2be72.slice/crio-a65e684207cd1948f4e112e46674a803750c10b353d15d5877e2b7f5c8987763 WatchSource:0}: Error finding container a65e684207cd1948f4e112e46674a803750c10b353d15d5877e2b7f5c8987763: Status 404 returned error can't find the container with id a65e684207cd1948f4e112e46674a803750c10b353d15d5877e2b7f5c8987763
Jul 13 18:33:56.898140 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:56.898105    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc4ffb449-cd8vx" event={"ID":"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72","Type":"ContainerStarted","Data":"efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47"}
Jul 13 18:33:56.898508 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:56.898179    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc4ffb449-cd8vx" event={"ID":"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72","Type":"ContainerStarted","Data":"a65e684207cd1948f4e112e46674a803750c10b353d15d5877e2b7f5c8987763"}
Jul 13 18:33:56.919339 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:33:56.919293    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cc4ffb449-cd8vx" podStartSLOduration=1.919279297 podStartE2EDuration="1.919279297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:33:56.917571096 +0000 UTC m=+118.048859557" watchObservedRunningTime="2026-07-13 18:33:56.919279297 +0000 UTC m=+118.050567757"
Jul 13 18:34:05.878710 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:05.878664    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:34:05.879209 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:05.878740    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:34:05.883538 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:05.883514    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:34:05.931965 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:05.931938    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:34:06.062710 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:06.062680    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fc545f97-w9tsl"]
Jul 13 18:34:09.173996 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:09.173964    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:34:09.176301 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:09.176282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698c185e-1b6a-43a7-97fb-725ad56c0962-metrics-certs\") pod \"network-metrics-daemon-zvvkh\" (UID: \"698c185e-1b6a-43a7-97fb-725ad56c0962\") " pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:34:09.364808 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:09.364783    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bwd4z\""
Jul 13 18:34:09.372157 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:09.372143    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zvvkh"
Jul 13 18:34:09.509296 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:09.509269    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zvvkh"]
Jul 13 18:34:09.512202 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:34:09.512158    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698c185e_1b6a_43a7_97fb_725ad56c0962.slice/crio-3277c758f8bede6fab328f7f6750f859a0142b0170b43e9dc6bab815fbe19062 WatchSource:0}: Error finding container 3277c758f8bede6fab328f7f6750f859a0142b0170b43e9dc6bab815fbe19062: Status 404 returned error can't find the container with id 3277c758f8bede6fab328f7f6750f859a0142b0170b43e9dc6bab815fbe19062
Jul 13 18:34:09.941819 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:09.941780    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zvvkh" event={"ID":"698c185e-1b6a-43a7-97fb-725ad56c0962","Type":"ContainerStarted","Data":"3277c758f8bede6fab328f7f6750f859a0142b0170b43e9dc6bab815fbe19062"}
Jul 13 18:34:10.946197 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:10.946168    2569 generic.go:356] "Generic (PLEG): container finished" podID="cc90ccc1-03cf-49db-bfd4-5458ec6e63a1" containerID="5d8f820d12819416d88111dc31be4e5cd81b439a016e9dfd1b490ca70ba3a169" exitCode=0
Jul 13 18:34:10.946618 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:10.946244    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps" event={"ID":"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1","Type":"ContainerDied","Data":"5d8f820d12819416d88111dc31be4e5cd81b439a016e9dfd1b490ca70ba3a169"}
Jul 13 18:34:10.946716 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:10.946642    2569 scope.go:117] "RemoveContainer" containerID="5d8f820d12819416d88111dc31be4e5cd81b439a016e9dfd1b490ca70ba3a169"
Jul 13 18:34:10.947995 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:10.947958    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zvvkh" event={"ID":"698c185e-1b6a-43a7-97fb-725ad56c0962","Type":"ContainerStarted","Data":"c62451a8d1bb5462b4d916cf22aafa49ab6f503e9a9f908b15c40593114ba827"}
Jul 13 18:34:10.948097 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:10.948015    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zvvkh" event={"ID":"698c185e-1b6a-43a7-97fb-725ad56c0962","Type":"ContainerStarted","Data":"21c576b3ba3a823e7135856e38ed69c12015af223a0a2989bf3394ad2f4cb51b"}
Jul 13 18:34:11.000507 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:11.000457    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zvvkh" podStartSLOduration=129.784684341 podStartE2EDuration="2m11.00044015s" totalImagesPullingTime="1.215755809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:34:11.000222456 +0000 UTC m=+132.131510942" watchObservedRunningTime="2026-07-13 18:34:11.00044015 +0000 UTC m=+132.131728615"
Jul 13 18:34:11.953010 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:11.952976    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-cndps" event={"ID":"cc90ccc1-03cf-49db-bfd4-5458ec6e63a1","Type":"ContainerStarted","Data":"2f6094237023d078daac445019bcd936c5a74e8e50a4b2c1588918d062c3ee11"}
Jul 13 18:34:31.082487 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.082441    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7fc545f97-w9tsl" podUID="d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" containerName="console" containerID="cri-o://22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04" gracePeriod=15
Jul 13 18:34:31.324546 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.324520    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:34:31.452548 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.452455    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-oauth-config\") pod \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") "
Jul 13 18:34:31.452548 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.452499    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-service-ca\") pod \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") "
Jul 13 18:34:31.452548 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.452533    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmhd7\" (UniqueName: \"kubernetes.io/projected/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-kube-api-access-kmhd7\") pod \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") "
Jul 13 18:34:31.452861 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.452567    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-serving-cert\") pod \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") "
Jul 13 18:34:31.452861 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.452586    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-config\") pod \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") "
Jul 13 18:34:31.452861 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.452614    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-oauth-serving-cert\") pod \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\" (UID: \"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be\") "
Jul 13 18:34:31.453068 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.453038    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-config" (OuterVolumeSpecName: "console-config") pod "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" (UID: "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:34:31.453122 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.453060    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-service-ca" (OuterVolumeSpecName: "service-ca") pod "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" (UID: "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:34:31.453122 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.453071    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" (UID: "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:34:31.454975 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.454948    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" (UID: "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:34:31.455090 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.455059    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" (UID: "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:34:31.455090 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.455058    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-kube-api-access-kmhd7" (OuterVolumeSpecName: "kube-api-access-kmhd7") pod "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" (UID: "d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be"). InnerVolumeSpecName "kube-api-access-kmhd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:34:31.553184 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.553147    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:34:31.553184 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.553175    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-oauth-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:34:31.553184 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.553184    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-oauth-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:34:31.553184 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.553193    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:34:31.553437 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.553202    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmhd7\" (UniqueName: \"kubernetes.io/projected/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-kube-api-access-kmhd7\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:34:31.553437 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:31.553210    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be-console-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:34:32.019532 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:32.019492    2569 generic.go:356] "Generic (PLEG): container finished" podID="d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" containerID="22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04" exitCode=0
Jul 13 18:34:32.019709 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:32.019551    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc545f97-w9tsl"
Jul 13 18:34:32.019709 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:32.019552    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc545f97-w9tsl" event={"ID":"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be","Type":"ContainerDied","Data":"22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04"}
Jul 13 18:34:32.019709 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:32.019631    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc545f97-w9tsl" event={"ID":"d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be","Type":"ContainerDied","Data":"3740dd117b6ae82c3c6e2d5f8fc50be05b2b6ef16383bb0dd5a8feb88ac3bb3f"}
Jul 13 18:34:32.019709 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:32.019675    2569 scope.go:117] "RemoveContainer" containerID="22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04"
Jul 13 18:34:32.027986 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:32.027968    2569 scope.go:117] "RemoveContainer" containerID="22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04"
Jul 13 18:34:32.028253 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:34:32.028234    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04\": container with ID starting with 22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04 not found: ID does not exist" containerID="22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04"
Jul 13 18:34:32.028303 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:32.028262    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04"} err="failed to get container status \"22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04\": rpc error: code = NotFound desc = could not find container \"22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04\": container with ID starting with 22d034298521b04dc3a172b383ed441e962d091e2ac8dadd56def848ef21fd04 not found: ID does not exist"
Jul 13 18:34:32.040844 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:32.040823    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fc545f97-w9tsl"]
Jul 13 18:34:32.046071 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:32.046049    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fc545f97-w9tsl"]
Jul 13 18:34:33.348671 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:34:33.348629    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" path="/var/lib/kubelet/pods/d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be/volumes"
Jul 13 18:35:01.900551 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.900515    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6786cddc49-n2g9f"]
Jul 13 18:35:01.901007 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.900770    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" containerName="console"
Jul 13 18:35:01.901007 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.900782    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" containerName="console"
Jul 13 18:35:01.901007 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.900828    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8b8fe07-f8c6-4ed3-bc43-f1e57b5679be" containerName="console"
Jul 13 18:35:01.904847 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.904822    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:01.925458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.925436    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6786cddc49-n2g9f"]
Jul 13 18:35:01.957513 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.957477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk868\" (UniqueName: \"kubernetes.io/projected/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-kube-api-access-kk868\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:01.957513 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.957513    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-oauth-config\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:01.957759 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.957533    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-oauth-serving-cert\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:01.957759 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.957571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-serving-cert\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:01.957759 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.957595    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-config\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:01.957759 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.957616    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-service-ca\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:01.957759 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:01.957630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-trusted-ca-bundle\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.058291 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.058260    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-oauth-serving-cert\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.058488 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.058302    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-serving-cert\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.058488 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.058431    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-config\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.058488 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.058481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-service-ca\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.058674 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.058508    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-trusted-ca-bundle\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.058674 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.058625    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk868\" (UniqueName: \"kubernetes.io/projected/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-kube-api-access-kk868\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.058778 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.058691    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-oauth-config\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.059144 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.059121    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-config\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.059264 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.059147    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-oauth-serving-cert\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.059367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.059345    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-service-ca\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.059437 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.059421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-trusted-ca-bundle\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.061013 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.060991    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-serving-cert\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.061152 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.061121    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-oauth-config\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.068557 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.068536    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk868\" (UniqueName: \"kubernetes.io/projected/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-kube-api-access-kk868\") pod \"console-6786cddc49-n2g9f\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") " pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.214068 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.213986    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:02.359406 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:02.359374    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6786cddc49-n2g9f"]
Jul 13 18:35:02.362941 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:35:02.362915    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b099ffc_969f_4840_ad1f_3c7d9c4d23b1.slice/crio-4c7405374b04cf3d575b4934b89086dcd586e9d1edafd4d5ced597d30809e02a WatchSource:0}: Error finding container 4c7405374b04cf3d575b4934b89086dcd586e9d1edafd4d5ced597d30809e02a: Status 404 returned error can't find the container with id 4c7405374b04cf3d575b4934b89086dcd586e9d1edafd4d5ced597d30809e02a
Jul 13 18:35:03.127153 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:03.127120    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6786cddc49-n2g9f" event={"ID":"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1","Type":"ContainerStarted","Data":"7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca"}
Jul 13 18:35:03.127519 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:03.127177    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6786cddc49-n2g9f" event={"ID":"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1","Type":"ContainerStarted","Data":"4c7405374b04cf3d575b4934b89086dcd586e9d1edafd4d5ced597d30809e02a"}
Jul 13 18:35:03.149841 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:03.149784    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6786cddc49-n2g9f" podStartSLOduration=2.14976786 podStartE2EDuration="2.14976786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:35:03.148925755 +0000 UTC m=+184.280214216" watchObservedRunningTime="2026-07-13 18:35:03.14976786 +0000 UTC m=+184.281056403"
Jul 13 18:35:12.215058 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:12.215023    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:12.215633 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:12.215085    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:12.219896 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:12.219873    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:13.165046 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:13.165015    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:35:13.228106 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:13.228074    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc4ffb449-cd8vx"]
Jul 13 18:35:38.249512 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.249472    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cc4ffb449-cd8vx" podUID="d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" containerName="console" containerID="cri-o://efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47" gracePeriod=15
Jul 13 18:35:38.491970 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.491948    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:35:38.621064 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.620978    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-config\") pod \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") "
Jul 13 18:35:38.621064 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.621029    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzvbw\" (UniqueName: \"kubernetes.io/projected/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-kube-api-access-nzvbw\") pod \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") "
Jul 13 18:35:38.621064 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.621053    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-oauth-config\") pod \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") "
Jul 13 18:35:38.621350 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.621083    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-oauth-serving-cert\") pod \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") "
Jul 13 18:35:38.621350 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.621116    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-trusted-ca-bundle\") pod \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") "
Jul 13 18:35:38.621350 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.621143    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-service-ca\") pod \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") "
Jul 13 18:35:38.621350 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.621186    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-serving-cert\") pod \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\" (UID: \"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72\") "
Jul 13 18:35:38.621518 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.621395    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-config" (OuterVolumeSpecName: "console-config") pod "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" (UID: "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:35:38.621765 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.621732    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" (UID: "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:35:38.621765 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.621741    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-service-ca" (OuterVolumeSpecName: "service-ca") pod "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" (UID: "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:35:38.621765 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.621753    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" (UID: "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:35:38.623292 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.623265    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-kube-api-access-nzvbw" (OuterVolumeSpecName: "kube-api-access-nzvbw") pod "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" (UID: "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72"). InnerVolumeSpecName "kube-api-access-nzvbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:35:38.623384 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.623353    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" (UID: "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:35:38.623422 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.623381    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" (UID: "d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:35:38.722335 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.722297    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzvbw\" (UniqueName: \"kubernetes.io/projected/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-kube-api-access-nzvbw\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:35:38.722335 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.722328    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-oauth-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:35:38.722335 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.722337    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-oauth-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:35:38.722335 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.722346    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-trusted-ca-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:35:38.722574 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.722356    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:35:38.722574 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.722364    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:35:38.722574 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:38.722373    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72-console-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:35:39.251242 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:39.251203    2569 generic.go:356] "Generic (PLEG): container finished" podID="d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" containerID="efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47" exitCode=0
Jul 13 18:35:39.251618 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:39.251249    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc4ffb449-cd8vx"
Jul 13 18:35:39.251618 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:39.251265    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc4ffb449-cd8vx" event={"ID":"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72","Type":"ContainerDied","Data":"efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47"}
Jul 13 18:35:39.251618 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:39.251316    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc4ffb449-cd8vx" event={"ID":"d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72","Type":"ContainerDied","Data":"a65e684207cd1948f4e112e46674a803750c10b353d15d5877e2b7f5c8987763"}
Jul 13 18:35:39.251618 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:39.251343    2569 scope.go:117] "RemoveContainer" containerID="efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47"
Jul 13 18:35:39.259791 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:39.259772    2569 scope.go:117] "RemoveContainer" containerID="efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47"
Jul 13 18:35:39.260033 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:35:39.260014    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47\": container with ID starting with efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47 not found: ID does not exist" containerID="efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47"
Jul 13 18:35:39.260075 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:39.260042    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47"} err="failed to get container status \"efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47\": rpc error: code = NotFound desc = could not find container \"efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47\": container with ID starting with efd559adf6c8282d58d309366dfc3c9b0ecf387b971350dfecfbe1f29113dc47 not found: ID does not exist"
Jul 13 18:35:39.274342 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:39.274315    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc4ffb449-cd8vx"]
Jul 13 18:35:39.277940 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:39.277919    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cc4ffb449-cd8vx"]
Jul 13 18:35:39.348274 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:35:39.348242    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" path="/var/lib/kubelet/pods/d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72/volumes"
Jul 13 18:36:09.083039 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.083006    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-588c585d96-kt6mk"]
Jul 13 18:36:09.083487 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.083257    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" containerName="console"
Jul 13 18:36:09.083487 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.083267    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" containerName="console"
Jul 13 18:36:09.083487 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.083315    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d34aa3fe-5cdc-4e1c-8d6a-0129f3c2be72" containerName="console"
Jul 13 18:36:09.087311 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.087295    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.098050 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.098026    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-588c585d96-kt6mk"]
Jul 13 18:36:09.256470 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.256426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-serving-cert\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.256470 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.256470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-trusted-ca-bundle\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.256702 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.256536    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9cn5\" (UniqueName: \"kubernetes.io/projected/6ab2262e-8852-47fa-8c9d-bca8f43f480f-kube-api-access-c9cn5\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.256702 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.256558    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-oauth-config\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.256702 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.256578    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-config\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.256702 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.256596    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-service-ca\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.256702 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.256615    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-oauth-serving-cert\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.357751 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.357645    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-oauth-config\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.357751 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.357710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-config\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.357751 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.357729    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-service-ca\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.357751 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.357746    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-oauth-serving-cert\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.358068 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.357769    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-serving-cert\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.358068 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.357786    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-trusted-ca-bundle\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.358068 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.357867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9cn5\" (UniqueName: \"kubernetes.io/projected/6ab2262e-8852-47fa-8c9d-bca8f43f480f-kube-api-access-c9cn5\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.364272 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.359024    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-oauth-serving-cert\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.364272 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.359024    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-config\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.364272 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.359294    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-service-ca\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.364272 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.359682    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-trusted-ca-bundle\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.364272 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.360813    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-serving-cert\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.364272 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.360852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-oauth-config\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.373890 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.373866    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9cn5\" (UniqueName: \"kubernetes.io/projected/6ab2262e-8852-47fa-8c9d-bca8f43f480f-kube-api-access-c9cn5\") pod \"console-588c585d96-kt6mk\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") " pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.397265 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.397238    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:09.530053 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:09.530022    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-588c585d96-kt6mk"]
Jul 13 18:36:09.532886 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:36:09.532862    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab2262e_8852_47fa_8c9d_bca8f43f480f.slice/crio-15dcb30493257a87e9ff366bea50d5c77680e763558d00ecc4014427a4bcb20c WatchSource:0}: Error finding container 15dcb30493257a87e9ff366bea50d5c77680e763558d00ecc4014427a4bcb20c: Status 404 returned error can't find the container with id 15dcb30493257a87e9ff366bea50d5c77680e763558d00ecc4014427a4bcb20c
Jul 13 18:36:10.356238 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:10.356197    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588c585d96-kt6mk" event={"ID":"6ab2262e-8852-47fa-8c9d-bca8f43f480f","Type":"ContainerStarted","Data":"a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936"}
Jul 13 18:36:10.356626 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:10.356250    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588c585d96-kt6mk" event={"ID":"6ab2262e-8852-47fa-8c9d-bca8f43f480f","Type":"ContainerStarted","Data":"15dcb30493257a87e9ff366bea50d5c77680e763558d00ecc4014427a4bcb20c"}
Jul 13 18:36:10.374067 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:10.374022    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-588c585d96-kt6mk" podStartSLOduration=1.37400661 podStartE2EDuration="1.37400661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:36:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:36:10.373933822 +0000 UTC m=+251.505222285" watchObservedRunningTime="2026-07-13 18:36:10.37400661 +0000 UTC m=+251.505295072"
Jul 13 18:36:19.397615 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:19.397574    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:19.398118 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:19.397638    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:19.402453 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:19.402430    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:20.393935 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:20.393905    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:36:20.448205 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:20.448173    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6786cddc49-n2g9f"]
Jul 13 18:36:45.475097 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.475041    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6786cddc49-n2g9f" podUID="0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" containerName="console" containerID="cri-o://7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca" gracePeriod=15
Jul 13 18:36:45.714181 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.714156    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:36:45.824849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.824746    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-oauth-config\") pod \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") "
Jul 13 18:36:45.824849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.824822    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-oauth-serving-cert\") pod \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") "
Jul 13 18:36:45.824849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.824857    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk868\" (UniqueName: \"kubernetes.io/projected/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-kube-api-access-kk868\") pod \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") "
Jul 13 18:36:45.825141 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.824883    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-serving-cert\") pod \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") "
Jul 13 18:36:45.825141 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.824916    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-service-ca\") pod \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") "
Jul 13 18:36:45.825141 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.824940    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-trusted-ca-bundle\") pod \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") "
Jul 13 18:36:45.825141 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.824955    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-config\") pod \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\" (UID: \"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1\") "
Jul 13 18:36:45.825382 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.825355    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" (UID: "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:36:45.825441 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.825374    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-service-ca" (OuterVolumeSpecName: "service-ca") pod "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" (UID: "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:36:45.825441 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.825418    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" (UID: "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:36:45.825539 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.825467    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-config" (OuterVolumeSpecName: "console-config") pod "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" (UID: "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:36:45.827250 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.827224    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-kube-api-access-kk868" (OuterVolumeSpecName: "kube-api-access-kk868") pod "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" (UID: "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1"). InnerVolumeSpecName "kube-api-access-kk868". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:36:45.827523 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.827501    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" (UID: "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:36:45.827622 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.827525    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" (UID: "0b099ffc-969f-4840-ad1f-3c7d9c4d23b1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:36:45.925825 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.925787    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:36:45.925825 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.925822    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-trusted-ca-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:36:45.925825 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.925832    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:36:45.926023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.925841    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-oauth-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:36:45.926023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.925850    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-oauth-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:36:45.926023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.925858    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk868\" (UniqueName: \"kubernetes.io/projected/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-kube-api-access-kk868\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:36:45.926023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:45.925867    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1-console-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:36:46.476623 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:46.476584    2569 generic.go:356] "Generic (PLEG): container finished" podID="0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" containerID="7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca" exitCode=0
Jul 13 18:36:46.477093 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:46.476630    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6786cddc49-n2g9f"
Jul 13 18:36:46.477093 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:46.476687    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6786cddc49-n2g9f" event={"ID":"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1","Type":"ContainerDied","Data":"7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca"}
Jul 13 18:36:46.477093 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:46.476749    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6786cddc49-n2g9f" event={"ID":"0b099ffc-969f-4840-ad1f-3c7d9c4d23b1","Type":"ContainerDied","Data":"4c7405374b04cf3d575b4934b89086dcd586e9d1edafd4d5ced597d30809e02a"}
Jul 13 18:36:46.477093 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:46.476778    2569 scope.go:117] "RemoveContainer" containerID="7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca"
Jul 13 18:36:46.485120 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:46.485103    2569 scope.go:117] "RemoveContainer" containerID="7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca"
Jul 13 18:36:46.485376 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:36:46.485360    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca\": container with ID starting with 7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca not found: ID does not exist" containerID="7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca"
Jul 13 18:36:46.485459 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:46.485389    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca"} err="failed to get container status \"7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca\": rpc error: code = NotFound desc = could not find container \"7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca\": container with ID starting with 7c646e41eeef5965928afe94462554375e95e1ab15fabd83b1bf1ba11d63f6ca not found: ID does not exist"
Jul 13 18:36:46.496925 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:46.496898    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6786cddc49-n2g9f"]
Jul 13 18:36:46.500616 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:46.500596    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6786cddc49-n2g9f"]
Jul 13 18:36:47.349299 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:47.349258    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" path="/var/lib/kubelet/pods/0b099ffc-969f-4840-ad1f-3c7d9c4d23b1/volumes"
Jul 13 18:36:59.220106 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:59.220076    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:36:59.222134 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:59.222115    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:36:59.223924 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:36:59.223902    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 18:37:13.446509 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.446476    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"]
Jul 13 18:37:13.448908 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.446721    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" containerName="console"
Jul 13 18:37:13.448908 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.446732    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" containerName="console"
Jul 13 18:37:13.448908 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.446789    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b099ffc-969f-4840-ad1f-3c7d9c4d23b1" containerName="console"
Jul 13 18:37:13.449770 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.449755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.452236 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.452212    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 13 18:37:13.452353 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.452222    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 13 18:37:13.453132 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.453117    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tjz8t\""
Jul 13 18:37:13.459078 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.459056    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"]
Jul 13 18:37:13.550200 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.550165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjfzk\" (UniqueName: \"kubernetes.io/projected/b1ac6c0f-7185-4965-ae93-8d05e2393f19-kube-api-access-rjfzk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.550390 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.550216    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.550390 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.550290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.650765 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.650734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjfzk\" (UniqueName: \"kubernetes.io/projected/b1ac6c0f-7185-4965-ae93-8d05e2393f19-kube-api-access-rjfzk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.650926 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.650778    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.650926 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.650819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.651128 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.651112    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.651168 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.651140    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.660115 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.660087    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjfzk\" (UniqueName: \"kubernetes.io/projected/b1ac6c0f-7185-4965-ae93-8d05e2393f19-kube-api-access-rjfzk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.758952 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.758920    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:13.885525 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.885488    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"]
Jul 13 18:37:13.888096 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:37:13.888073    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ac6c0f_7185_4965_ae93_8d05e2393f19.slice/crio-991fd980f1884da09673d8aab195c0fafdff7fd104193abf8de7dc60a6a58b61 WatchSource:0}: Error finding container 991fd980f1884da09673d8aab195c0fafdff7fd104193abf8de7dc60a6a58b61: Status 404 returned error can't find the container with id 991fd980f1884da09673d8aab195c0fafdff7fd104193abf8de7dc60a6a58b61
Jul 13 18:37:13.890069 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:13.890053    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 18:37:14.571264 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:14.571226    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s" event={"ID":"b1ac6c0f-7185-4965-ae93-8d05e2393f19","Type":"ContainerStarted","Data":"991fd980f1884da09673d8aab195c0fafdff7fd104193abf8de7dc60a6a58b61"}
Jul 13 18:37:22.601240 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:22.601204    2569 generic.go:356] "Generic (PLEG): container finished" podID="b1ac6c0f-7185-4965-ae93-8d05e2393f19" containerID="d3f16e85f3e1bc0a1cc37fcc605f2214945f30f10cd1530cd0482368febc213b" exitCode=0
Jul 13 18:37:22.601782 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:22.601250    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s" event={"ID":"b1ac6c0f-7185-4965-ae93-8d05e2393f19","Type":"ContainerDied","Data":"d3f16e85f3e1bc0a1cc37fcc605f2214945f30f10cd1530cd0482368febc213b"}
Jul 13 18:37:25.613125 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:25.613092    2569 generic.go:356] "Generic (PLEG): container finished" podID="b1ac6c0f-7185-4965-ae93-8d05e2393f19" containerID="bdfb03b866b7450c8bd6c55e2c380b26698fcc3ea9b4337a080aa4c5838d5254" exitCode=0
Jul 13 18:37:25.613522 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:25.613188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s" event={"ID":"b1ac6c0f-7185-4965-ae93-8d05e2393f19","Type":"ContainerDied","Data":"bdfb03b866b7450c8bd6c55e2c380b26698fcc3ea9b4337a080aa4c5838d5254"}
Jul 13 18:37:35.650539 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:35.650507    2569 generic.go:356] "Generic (PLEG): container finished" podID="b1ac6c0f-7185-4965-ae93-8d05e2393f19" containerID="902bae331f180f7dc249d69d6dae3a35b37775517dc05c81991a6052e589625f" exitCode=0
Jul 13 18:37:35.651023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:35.650570    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s" event={"ID":"b1ac6c0f-7185-4965-ae93-8d05e2393f19","Type":"ContainerDied","Data":"902bae331f180f7dc249d69d6dae3a35b37775517dc05c81991a6052e589625f"}
Jul 13 18:37:36.775412 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:36.775389    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:36.831445 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:36.831421    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-util\") pod \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") "
Jul 13 18:37:36.831599 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:36.831479    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-bundle\") pod \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") "
Jul 13 18:37:36.831599 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:36.831507    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjfzk\" (UniqueName: \"kubernetes.io/projected/b1ac6c0f-7185-4965-ae93-8d05e2393f19-kube-api-access-rjfzk\") pod \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\" (UID: \"b1ac6c0f-7185-4965-ae93-8d05e2393f19\") "
Jul 13 18:37:36.832264 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:36.832225    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-bundle" (OuterVolumeSpecName: "bundle") pod "b1ac6c0f-7185-4965-ae93-8d05e2393f19" (UID: "b1ac6c0f-7185-4965-ae93-8d05e2393f19"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:37:36.833790 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:36.833767    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ac6c0f-7185-4965-ae93-8d05e2393f19-kube-api-access-rjfzk" (OuterVolumeSpecName: "kube-api-access-rjfzk") pod "b1ac6c0f-7185-4965-ae93-8d05e2393f19" (UID: "b1ac6c0f-7185-4965-ae93-8d05e2393f19"). InnerVolumeSpecName "kube-api-access-rjfzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:37:36.836422 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:36.836403    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-util" (OuterVolumeSpecName: "util") pod "b1ac6c0f-7185-4965-ae93-8d05e2393f19" (UID: "b1ac6c0f-7185-4965-ae93-8d05e2393f19"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:37:36.931998 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:36.931918    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:37:36.931998 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:36.931945    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjfzk\" (UniqueName: \"kubernetes.io/projected/b1ac6c0f-7185-4965-ae93-8d05e2393f19-kube-api-access-rjfzk\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:37:36.931998 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:36.931954    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1ac6c0f-7185-4965-ae93-8d05e2393f19-util\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:37:37.659900 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:37.659820    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s"
Jul 13 18:37:37.659900 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:37.659829    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ndn2s" event={"ID":"b1ac6c0f-7185-4965-ae93-8d05e2393f19","Type":"ContainerDied","Data":"991fd980f1884da09673d8aab195c0fafdff7fd104193abf8de7dc60a6a58b61"}
Jul 13 18:37:37.659900 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:37.659899    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="991fd980f1884da09673d8aab195c0fafdff7fd104193abf8de7dc60a6a58b61"
Jul 13 18:37:41.459333 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.459288    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"]
Jul 13 18:37:41.459810 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.459535    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ac6c0f-7185-4965-ae93-8d05e2393f19" containerName="util"
Jul 13 18:37:41.459810 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.459546    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ac6c0f-7185-4965-ae93-8d05e2393f19" containerName="util"
Jul 13 18:37:41.459810 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.459556    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ac6c0f-7185-4965-ae93-8d05e2393f19" containerName="pull"
Jul 13 18:37:41.459810 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.459561    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ac6c0f-7185-4965-ae93-8d05e2393f19" containerName="pull"
Jul 13 18:37:41.459810 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.459569    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ac6c0f-7185-4965-ae93-8d05e2393f19" containerName="extract"
Jul 13 18:37:41.459810 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.459574    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ac6c0f-7185-4965-ae93-8d05e2393f19" containerName="extract"
Jul 13 18:37:41.459810 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.459622    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1ac6c0f-7185-4965-ae93-8d05e2393f19" containerName="extract"
Jul 13 18:37:41.462511 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.462490    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"
Jul 13 18:37:41.465942 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.465925    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lg4sg\""
Jul 13 18:37:41.466134 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.466122    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 13 18:37:41.466779 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.466763    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 13 18:37:41.485491 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.485471    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"]
Jul 13 18:37:41.562995 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.562963    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9j2w\" (UniqueName: \"kubernetes.io/projected/8c7d10b0-4d42-4286-aab3-932561995d09-kube-api-access-b9j2w\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqwvv\" (UID: \"8c7d10b0-4d42-4286-aab3-932561995d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"
Jul 13 18:37:41.563166 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.563015    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7d10b0-4d42-4286-aab3-932561995d09-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqwvv\" (UID: \"8c7d10b0-4d42-4286-aab3-932561995d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"
Jul 13 18:37:41.664402 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.664363    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9j2w\" (UniqueName: \"kubernetes.io/projected/8c7d10b0-4d42-4286-aab3-932561995d09-kube-api-access-b9j2w\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqwvv\" (UID: \"8c7d10b0-4d42-4286-aab3-932561995d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"
Jul 13 18:37:41.664543 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.664416    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7d10b0-4d42-4286-aab3-932561995d09-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqwvv\" (UID: \"8c7d10b0-4d42-4286-aab3-932561995d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"
Jul 13 18:37:41.664781 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.664764    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7d10b0-4d42-4286-aab3-932561995d09-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqwvv\" (UID: \"8c7d10b0-4d42-4286-aab3-932561995d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"
Jul 13 18:37:41.673396 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.673362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9j2w\" (UniqueName: \"kubernetes.io/projected/8c7d10b0-4d42-4286-aab3-932561995d09-kube-api-access-b9j2w\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqwvv\" (UID: \"8c7d10b0-4d42-4286-aab3-932561995d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"
Jul 13 18:37:41.770903 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.770863    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"
Jul 13 18:37:41.908374 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:41.908333    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv"]
Jul 13 18:37:41.912600 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:37:41.912559    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7d10b0_4d42_4286_aab3_932561995d09.slice/crio-a98bedb8679b5a22fefdd79dfec49cfe795d3cb4b5fbf507bce23b286dbffe33 WatchSource:0}: Error finding container a98bedb8679b5a22fefdd79dfec49cfe795d3cb4b5fbf507bce23b286dbffe33: Status 404 returned error can't find the container with id a98bedb8679b5a22fefdd79dfec49cfe795d3cb4b5fbf507bce23b286dbffe33
Jul 13 18:37:42.678221 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:42.678186    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv" event={"ID":"8c7d10b0-4d42-4286-aab3-932561995d09","Type":"ContainerStarted","Data":"a98bedb8679b5a22fefdd79dfec49cfe795d3cb4b5fbf507bce23b286dbffe33"}
Jul 13 18:37:44.687328 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:44.687294    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv" event={"ID":"8c7d10b0-4d42-4286-aab3-932561995d09","Type":"ContainerStarted","Data":"8effd7ec3144d18ea5d6e77d2a9136566845029285e1df64872c8a0d733819b5"}
Jul 13 18:37:44.731881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:44.731820    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqwvv" podStartSLOduration=1.836320394 podStartE2EDuration="3.7318032s" totalImagesPullingTime="1.895482806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:37:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:37:44.729186907 +0000 UTC m=+345.860475369" watchObservedRunningTime="2026-07-13 18:37:44.7318032 +0000 UTC m=+345.863091659"
Jul 13 18:37:45.922340 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:45.922310    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"]
Jul 13 18:37:45.925614 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:45.925588    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:45.928321 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:45.928299    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tjz8t\""
Jul 13 18:37:45.928444 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:45.928320    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 13 18:37:45.929997 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:45.929978    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 13 18:37:45.938632 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:45.938595    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"]
Jul 13 18:37:45.997817 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:45.997786    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:45.997981 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:45.997831    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:45.997981 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:45.997886    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzqbg\" (UniqueName: \"kubernetes.io/projected/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-kube-api-access-nzqbg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:46.098710 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.098677    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:46.098862 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.098728    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:46.098862 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.098756    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzqbg\" (UniqueName: \"kubernetes.io/projected/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-kube-api-access-nzqbg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:46.099059 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.099039    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:46.099097 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.099076    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:46.117133 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.117101    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzqbg\" (UniqueName: \"kubernetes.io/projected/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-kube-api-access-nzqbg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:46.234550 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.234526    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:46.369544 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.369514    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"]
Jul 13 18:37:46.372851 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:37:46.372806    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa2dd5a2_7e47_40f5_9418_c4dd1731f3bd.slice/crio-4a3f234e07e0c4be8edfff9d498c79d391df975ab375577bdd32b5d0e9f9ef51 WatchSource:0}: Error finding container 4a3f234e07e0c4be8edfff9d498c79d391df975ab375577bdd32b5d0e9f9ef51: Status 404 returned error can't find the container with id 4a3f234e07e0c4be8edfff9d498c79d391df975ab375577bdd32b5d0e9f9ef51
Jul 13 18:37:46.696819 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.696726    2569 generic.go:356] "Generic (PLEG): container finished" podID="fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" containerID="06d41e6338365ef8119348c135f590000ddfda7c3b8f79ea615e51b157d7490e" exitCode=0
Jul 13 18:37:46.696819 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.696779    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4" event={"ID":"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd","Type":"ContainerDied","Data":"06d41e6338365ef8119348c135f590000ddfda7c3b8f79ea615e51b157d7490e"}
Jul 13 18:37:46.697129 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:46.696825    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4" event={"ID":"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd","Type":"ContainerStarted","Data":"4a3f234e07e0c4be8edfff9d498c79d391df975ab375577bdd32b5d0e9f9ef51"}
Jul 13 18:37:49.712770 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:49.712670    2569 generic.go:356] "Generic (PLEG): container finished" podID="fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" containerID="2cd11f55684a52bbbbbcc08c6a68e398b86a271cbeebeb139e2c05978474f557" exitCode=0
Jul 13 18:37:49.713216 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:49.712760    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4" event={"ID":"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd","Type":"ContainerDied","Data":"2cd11f55684a52bbbbbcc08c6a68e398b86a271cbeebeb139e2c05978474f557"}
Jul 13 18:37:50.718610 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:50.718577    2569 generic.go:356] "Generic (PLEG): container finished" podID="fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" containerID="f7b922c56cf0bdeb329b4d2bbd6203d8820ab1b016e0533a828b98d9c6583576" exitCode=0
Jul 13 18:37:50.718980 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:50.718627    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4" event={"ID":"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd","Type":"ContainerDied","Data":"f7b922c56cf0bdeb329b4d2bbd6203d8820ab1b016e0533a828b98d9c6583576"}
Jul 13 18:37:51.846219 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:51.846197    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:51.949226 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:51.949196    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-util\") pod \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") "
Jul 13 18:37:51.949400 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:51.949255    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-bundle\") pod \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") "
Jul 13 18:37:51.949400 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:51.949283    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzqbg\" (UniqueName: \"kubernetes.io/projected/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-kube-api-access-nzqbg\") pod \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\" (UID: \"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd\") "
Jul 13 18:37:51.949671 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:51.949628    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-bundle" (OuterVolumeSpecName: "bundle") pod "fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" (UID: "fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:37:51.951569 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:51.951550    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-kube-api-access-nzqbg" (OuterVolumeSpecName: "kube-api-access-nzqbg") pod "fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" (UID: "fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd"). InnerVolumeSpecName "kube-api-access-nzqbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:37:51.953870 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:51.953847    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-util" (OuterVolumeSpecName: "util") pod "fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" (UID: "fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:37:52.050081 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.050047    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-util\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:37:52.050081 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.050077    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:37:52.050081 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.050086    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzqbg\" (UniqueName: \"kubernetes.io/projected/fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd-kube-api-access-nzqbg\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:37:52.728469 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.728435    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4" event={"ID":"fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd","Type":"ContainerDied","Data":"4a3f234e07e0c4be8edfff9d498c79d391df975ab375577bdd32b5d0e9f9ef51"}
Jul 13 18:37:52.728615 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.728485    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a3f234e07e0c4be8edfff9d498c79d391df975ab375577bdd32b5d0e9f9ef51"
Jul 13 18:37:52.728615 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.728486    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmlcs4"
Jul 13 18:37:52.784095 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.784062    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"]
Jul 13 18:37:52.784349 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.784337    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" containerName="pull"
Jul 13 18:37:52.784391 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.784353    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" containerName="pull"
Jul 13 18:37:52.784391 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.784370    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" containerName="extract"
Jul 13 18:37:52.784391 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.784376    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" containerName="extract"
Jul 13 18:37:52.784391 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.784385    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" containerName="util"
Jul 13 18:37:52.784391 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.784390    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" containerName="util"
Jul 13 18:37:52.784541 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.784437    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa2dd5a2-7e47-40f5-9418-c4dd1731f3bd" containerName="extract"
Jul 13 18:37:52.788645 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.788631    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"
Jul 13 18:37:52.791486 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.791462    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 13 18:37:52.791623 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.791516    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ksjrb\""
Jul 13 18:37:52.791623 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.791516    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 13 18:37:52.797514 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.797493    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"]
Jul 13 18:37:52.855714 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.855673    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f28n6\" (UniqueName: \"kubernetes.io/projected/229f3476-469b-4646-adcf-f01aac8d7675-kube-api-access-f28n6\") pod \"cert-manager-cainjector-559cdd8c87-sqktb\" (UID: \"229f3476-469b-4646-adcf-f01aac8d7675\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"
Jul 13 18:37:52.856080 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.855744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/229f3476-469b-4646-adcf-f01aac8d7675-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sqktb\" (UID: \"229f3476-469b-4646-adcf-f01aac8d7675\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"
Jul 13 18:37:52.956118 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.956085    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f28n6\" (UniqueName: \"kubernetes.io/projected/229f3476-469b-4646-adcf-f01aac8d7675-kube-api-access-f28n6\") pod \"cert-manager-cainjector-559cdd8c87-sqktb\" (UID: \"229f3476-469b-4646-adcf-f01aac8d7675\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"
Jul 13 18:37:52.956319 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.956137    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/229f3476-469b-4646-adcf-f01aac8d7675-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sqktb\" (UID: \"229f3476-469b-4646-adcf-f01aac8d7675\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"
Jul 13 18:37:52.969953 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.969927    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f28n6\" (UniqueName: \"kubernetes.io/projected/229f3476-469b-4646-adcf-f01aac8d7675-kube-api-access-f28n6\") pod \"cert-manager-cainjector-559cdd8c87-sqktb\" (UID: \"229f3476-469b-4646-adcf-f01aac8d7675\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"
Jul 13 18:37:52.970081 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:52.969929    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/229f3476-469b-4646-adcf-f01aac8d7675-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sqktb\" (UID: \"229f3476-469b-4646-adcf-f01aac8d7675\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"
Jul 13 18:37:53.104128 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:53.104036    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"
Jul 13 18:37:53.235835 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:53.235810    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sqktb"]
Jul 13 18:37:53.237966 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:37:53.237939    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod229f3476_469b_4646_adcf_f01aac8d7675.slice/crio-2ed293542bc8dbdcfc266251f357cd9d3f1e75e4846e7138e8de8ee8210cff11 WatchSource:0}: Error finding container 2ed293542bc8dbdcfc266251f357cd9d3f1e75e4846e7138e8de8ee8210cff11: Status 404 returned error can't find the container with id 2ed293542bc8dbdcfc266251f357cd9d3f1e75e4846e7138e8de8ee8210cff11
Jul 13 18:37:53.733125 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:53.733093    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb" event={"ID":"229f3476-469b-4646-adcf-f01aac8d7675","Type":"ContainerStarted","Data":"2ed293542bc8dbdcfc266251f357cd9d3f1e75e4846e7138e8de8ee8210cff11"}
Jul 13 18:37:55.925126 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:55.925088    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9pqfb"]
Jul 13 18:37:55.933031 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:55.933003    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9pqfb"
Jul 13 18:37:55.935637 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:55.935562    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-g4w89\""
Jul 13 18:37:55.939592 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:55.939546    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9pqfb"]
Jul 13 18:37:56.081260 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:56.081227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30bf01d8-e2e4-41a1-81b2-068ad6dc3a96-bound-sa-token\") pod \"cert-manager-55b4d954d5-9pqfb\" (UID: \"30bf01d8-e2e4-41a1-81b2-068ad6dc3a96\") " pod="cert-manager/cert-manager-55b4d954d5-9pqfb"
Jul 13 18:37:56.081435 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:56.081302    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj26w\" (UniqueName: \"kubernetes.io/projected/30bf01d8-e2e4-41a1-81b2-068ad6dc3a96-kube-api-access-lj26w\") pod \"cert-manager-55b4d954d5-9pqfb\" (UID: \"30bf01d8-e2e4-41a1-81b2-068ad6dc3a96\") " pod="cert-manager/cert-manager-55b4d954d5-9pqfb"
Jul 13 18:37:56.182232 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:56.182144    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30bf01d8-e2e4-41a1-81b2-068ad6dc3a96-bound-sa-token\") pod \"cert-manager-55b4d954d5-9pqfb\" (UID: \"30bf01d8-e2e4-41a1-81b2-068ad6dc3a96\") " pod="cert-manager/cert-manager-55b4d954d5-9pqfb"
Jul 13 18:37:56.182232 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:56.182184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj26w\" (UniqueName: \"kubernetes.io/projected/30bf01d8-e2e4-41a1-81b2-068ad6dc3a96-kube-api-access-lj26w\") pod \"cert-manager-55b4d954d5-9pqfb\" (UID: \"30bf01d8-e2e4-41a1-81b2-068ad6dc3a96\") " pod="cert-manager/cert-manager-55b4d954d5-9pqfb"
Jul 13 18:37:56.191754 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:56.191719    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30bf01d8-e2e4-41a1-81b2-068ad6dc3a96-bound-sa-token\") pod \"cert-manager-55b4d954d5-9pqfb\" (UID: \"30bf01d8-e2e4-41a1-81b2-068ad6dc3a96\") " pod="cert-manager/cert-manager-55b4d954d5-9pqfb"
Jul 13 18:37:56.192223 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:56.192193    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj26w\" (UniqueName: \"kubernetes.io/projected/30bf01d8-e2e4-41a1-81b2-068ad6dc3a96-kube-api-access-lj26w\") pod \"cert-manager-55b4d954d5-9pqfb\" (UID: \"30bf01d8-e2e4-41a1-81b2-068ad6dc3a96\") " pod="cert-manager/cert-manager-55b4d954d5-9pqfb"
Jul 13 18:37:56.246060 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:56.246023    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9pqfb"
Jul 13 18:37:56.733109 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:56.733081    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9pqfb"]
Jul 13 18:37:56.736299 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:37:56.736273    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30bf01d8_e2e4_41a1_81b2_068ad6dc3a96.slice/crio-8067e0579b624db5b6e149405cb9f7ad7f5631b97e4a3e089243149c2265b4b3 WatchSource:0}: Error finding container 8067e0579b624db5b6e149405cb9f7ad7f5631b97e4a3e089243149c2265b4b3: Status 404 returned error can't find the container with id 8067e0579b624db5b6e149405cb9f7ad7f5631b97e4a3e089243149c2265b4b3
Jul 13 18:37:56.748616 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:56.748582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9pqfb" event={"ID":"30bf01d8-e2e4-41a1-81b2-068ad6dc3a96","Type":"ContainerStarted","Data":"8067e0579b624db5b6e149405cb9f7ad7f5631b97e4a3e089243149c2265b4b3"}
Jul 13 18:37:57.753927 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:57.753891    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb" event={"ID":"229f3476-469b-4646-adcf-f01aac8d7675","Type":"ContainerStarted","Data":"9f4ab553bf935253eea13a0c2ee4e7409bc0b62b407f61e98e7454b6306b0209"}
Jul 13 18:37:57.755315 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:57.755290    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9pqfb" event={"ID":"30bf01d8-e2e4-41a1-81b2-068ad6dc3a96","Type":"ContainerStarted","Data":"0db4510ec48d1d8bca04213ac4e5f1752d133b1bf877ffc3a4541b8bae9d97ab"}
Jul 13 18:37:57.770523 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:57.770465    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sqktb" podStartSLOduration=2.333228766 podStartE2EDuration="5.770450701s" totalImagesPullingTime="3.437221935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:37:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:37:57.770005653 +0000 UTC m=+358.901294124" watchObservedRunningTime="2026-07-13 18:37:57.770450701 +0000 UTC m=+358.901739161"
Jul 13 18:37:57.786734 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:37:57.786639    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9pqfb" podStartSLOduration=2.786618827 podStartE2EDuration="2.786618827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:37:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:37:57.784945613 +0000 UTC m=+358.916234075" watchObservedRunningTime="2026-07-13 18:37:57.786618827 +0000 UTC m=+358.917907288"
Jul 13 18:38:12.793325 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:12.793292    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"]
Jul 13 18:38:12.795736 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:12.795720    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:12.798427 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:12.798406    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tjz8t\""
Jul 13 18:38:12.798427 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:12.798423    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 13 18:38:12.799251 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:12.799234    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 13 18:38:12.805739 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:12.805720    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"]
Jul 13 18:38:12.919992 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:12.919951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h46vb\" (UniqueName: \"kubernetes.io/projected/54c57520-533a-4382-a305-42778a34d594-kube-api-access-h46vb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:12.920162 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:12.920017    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:12.920162 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:12.920056    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:13.020791 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.020752    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:13.020940 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.020806    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:13.020940 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.020838    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h46vb\" (UniqueName: \"kubernetes.io/projected/54c57520-533a-4382-a305-42778a34d594-kube-api-access-h46vb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:13.021131 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.021112    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:13.021172 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.021156    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:13.030918 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.030888    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h46vb\" (UniqueName: \"kubernetes.io/projected/54c57520-533a-4382-a305-42778a34d594-kube-api-access-h46vb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:13.104566 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.104481    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:13.232330 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.232302    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"]
Jul 13 18:38:13.234745 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:38:13.234713    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c57520_533a_4382_a305_42778a34d594.slice/crio-6ef3bb7e640a2be011e590b9d385dae7d588f7bf7d7d245d145730e2231db278 WatchSource:0}: Error finding container 6ef3bb7e640a2be011e590b9d385dae7d588f7bf7d7d245d145730e2231db278: Status 404 returned error can't find the container with id 6ef3bb7e640a2be011e590b9d385dae7d588f7bf7d7d245d145730e2231db278
Jul 13 18:38:13.821570 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.821539    2569 generic.go:356] "Generic (PLEG): container finished" podID="54c57520-533a-4382-a305-42778a34d594" containerID="8fc4745c127a9314ff766ad7fd627fa75fde16e74ec583484ccbf74da48be1f2" exitCode=0
Jul 13 18:38:13.822000 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.821583    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt" event={"ID":"54c57520-533a-4382-a305-42778a34d594","Type":"ContainerDied","Data":"8fc4745c127a9314ff766ad7fd627fa75fde16e74ec583484ccbf74da48be1f2"}
Jul 13 18:38:13.822000 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:13.821619    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt" event={"ID":"54c57520-533a-4382-a305-42778a34d594","Type":"ContainerStarted","Data":"6ef3bb7e640a2be011e590b9d385dae7d588f7bf7d7d245d145730e2231db278"}
Jul 13 18:38:15.830682 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:15.830637    2569 generic.go:356] "Generic (PLEG): container finished" podID="54c57520-533a-4382-a305-42778a34d594" containerID="a00705b3a71ff7f77dc11ac20051e0d8a13599870a2d2bdfb1d3aae0c762acb0" exitCode=0
Jul 13 18:38:15.831076 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:15.830705    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt" event={"ID":"54c57520-533a-4382-a305-42778a34d594","Type":"ContainerDied","Data":"a00705b3a71ff7f77dc11ac20051e0d8a13599870a2d2bdfb1d3aae0c762acb0"}
Jul 13 18:38:16.836449 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:16.836411    2569 generic.go:356] "Generic (PLEG): container finished" podID="54c57520-533a-4382-a305-42778a34d594" containerID="1ebfa7d0761897cf74d8c14f6778cd531edba3cf29b41bab9e8b21fcd9c873cd" exitCode=0
Jul 13 18:38:16.836875 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:16.836500    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt" event={"ID":"54c57520-533a-4382-a305-42778a34d594","Type":"ContainerDied","Data":"1ebfa7d0761897cf74d8c14f6778cd531edba3cf29b41bab9e8b21fcd9c873cd"}
Jul 13 18:38:17.981132 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:17.981107    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:18.162080 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.161987    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-bundle\") pod \"54c57520-533a-4382-a305-42778a34d594\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") "
Jul 13 18:38:18.162080 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.162034    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h46vb\" (UniqueName: \"kubernetes.io/projected/54c57520-533a-4382-a305-42778a34d594-kube-api-access-h46vb\") pod \"54c57520-533a-4382-a305-42778a34d594\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") "
Jul 13 18:38:18.162080 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.162069    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-util\") pod \"54c57520-533a-4382-a305-42778a34d594\" (UID: \"54c57520-533a-4382-a305-42778a34d594\") "
Jul 13 18:38:18.162953 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.162918    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-bundle" (OuterVolumeSpecName: "bundle") pod "54c57520-533a-4382-a305-42778a34d594" (UID: "54c57520-533a-4382-a305-42778a34d594"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:38:18.164288 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.164256    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54c57520-533a-4382-a305-42778a34d594-kube-api-access-h46vb" (OuterVolumeSpecName: "kube-api-access-h46vb") pod "54c57520-533a-4382-a305-42778a34d594" (UID: "54c57520-533a-4382-a305-42778a34d594"). InnerVolumeSpecName "kube-api-access-h46vb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:38:18.167998 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.167977    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-util" (OuterVolumeSpecName: "util") pod "54c57520-533a-4382-a305-42778a34d594" (UID: "54c57520-533a-4382-a305-42778a34d594"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:38:18.262951 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.262910    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-util\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:38:18.262951 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.262944    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c57520-533a-4382-a305-42778a34d594-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:38:18.262951 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.262957    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h46vb\" (UniqueName: \"kubernetes.io/projected/54c57520-533a-4382-a305-42778a34d594-kube-api-access-h46vb\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:38:18.846950 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.846915    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt" event={"ID":"54c57520-533a-4382-a305-42778a34d594","Type":"ContainerDied","Data":"6ef3bb7e640a2be011e590b9d385dae7d588f7bf7d7d245d145730e2231db278"}
Jul 13 18:38:18.847119 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.846962    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ef3bb7e640a2be011e590b9d385dae7d588f7bf7d7d245d145730e2231db278"
Jul 13 18:38:18.847119 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:18.847005    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxsztt"
Jul 13 18:38:29.780422 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.780384    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"]
Jul 13 18:38:29.780881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.780688    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c57520-533a-4382-a305-42778a34d594" containerName="extract"
Jul 13 18:38:29.780881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.780701    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c57520-533a-4382-a305-42778a34d594" containerName="extract"
Jul 13 18:38:29.780881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.780715    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c57520-533a-4382-a305-42778a34d594" containerName="pull"
Jul 13 18:38:29.780881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.780720    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c57520-533a-4382-a305-42778a34d594" containerName="pull"
Jul 13 18:38:29.780881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.780727    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c57520-533a-4382-a305-42778a34d594" containerName="util"
Jul 13 18:38:29.780881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.780733    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c57520-533a-4382-a305-42778a34d594" containerName="util"
Jul 13 18:38:29.780881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.780780    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c57520-533a-4382-a305-42778a34d594" containerName="extract"
Jul 13 18:38:29.786690 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.786644    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.789271 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.789221    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 13 18:38:29.789407 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.789287    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 13 18:38:29.789407 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.789317    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 13 18:38:29.789524 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.789508    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vdvqz\""
Jul 13 18:38:29.789753 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.789741    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 13 18:38:29.806296 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.806273    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"]
Jul 13 18:38:29.859617 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.859590    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/76011d1d-de12-4769-b91f-e5600b77090a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.859760 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.859633    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76011d1d-de12-4769-b91f-e5600b77090a-tmp\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.859760 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.859693    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlltm\" (UniqueName: \"kubernetes.io/projected/76011d1d-de12-4769-b91f-e5600b77090a-kube-api-access-rlltm\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.859760 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.859756    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/76011d1d-de12-4769-b91f-e5600b77090a-manifests\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.859864 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.859776    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/76011d1d-de12-4769-b91f-e5600b77090a-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.960799 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.960764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/76011d1d-de12-4769-b91f-e5600b77090a-manifests\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.960955 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.960801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/76011d1d-de12-4769-b91f-e5600b77090a-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.960955 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.960841    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/76011d1d-de12-4769-b91f-e5600b77090a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.960955 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.960881    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76011d1d-de12-4769-b91f-e5600b77090a-tmp\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.960955 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.960905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlltm\" (UniqueName: \"kubernetes.io/projected/76011d1d-de12-4769-b91f-e5600b77090a-kube-api-access-rlltm\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.961234 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.961211    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/76011d1d-de12-4769-b91f-e5600b77090a-manifests\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.963313 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.963290    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76011d1d-de12-4769-b91f-e5600b77090a-tmp\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.963447 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.963420    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/76011d1d-de12-4769-b91f-e5600b77090a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.963508 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.963494    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/76011d1d-de12-4769-b91f-e5600b77090a-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:29.971030 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:29.971006    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlltm\" (UniqueName: \"kubernetes.io/projected/76011d1d-de12-4769-b91f-e5600b77090a-kube-api-access-rlltm\") pod \"opendatahub-operator-controller-manager-b4789cfbf-t92zq\" (UID: \"76011d1d-de12-4769-b91f-e5600b77090a\") " pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:30.096287 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:30.096206    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:30.237846 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:30.237816    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"]
Jul 13 18:38:30.240984 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:38:30.240959    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76011d1d_de12_4769_b91f_e5600b77090a.slice/crio-fe5a2787ec1b5589a4bc1c2b1c0c1c8be7153753ee7bdf24c4ae63848f7e3f6a WatchSource:0}: Error finding container fe5a2787ec1b5589a4bc1c2b1c0c1c8be7153753ee7bdf24c4ae63848f7e3f6a: Status 404 returned error can't find the container with id fe5a2787ec1b5589a4bc1c2b1c0c1c8be7153753ee7bdf24c4ae63848f7e3f6a
Jul 13 18:38:30.894478 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:30.894443    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq" event={"ID":"76011d1d-de12-4769-b91f-e5600b77090a","Type":"ContainerStarted","Data":"fe5a2787ec1b5589a4bc1c2b1c0c1c8be7153753ee7bdf24c4ae63848f7e3f6a"}
Jul 13 18:38:33.910293 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:33.910260    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq" event={"ID":"76011d1d-de12-4769-b91f-e5600b77090a","Type":"ContainerStarted","Data":"38ee084d4df166f19f16a59e717c5e123629f9fd096b9cd7c6bba8149b16ab9f"}
Jul 13 18:38:34.915736 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:34.915699    2569 generic.go:356] "Generic (PLEG): container finished" podID="76011d1d-de12-4769-b91f-e5600b77090a" containerID="38ee084d4df166f19f16a59e717c5e123629f9fd096b9cd7c6bba8149b16ab9f" exitCode=0
Jul 13 18:38:34.916168 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:34.915796    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq" event={"ID":"76011d1d-de12-4769-b91f-e5600b77090a","Type":"ContainerDied","Data":"38ee084d4df166f19f16a59e717c5e123629f9fd096b9cd7c6bba8149b16ab9f"}
Jul 13 18:38:35.922401 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:35.922368    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq" event={"ID":"76011d1d-de12-4769-b91f-e5600b77090a","Type":"ContainerStarted","Data":"da60f585c753f7d01488cf884021ab4335599b6047749db94912b3f9df6d07e4"}
Jul 13 18:38:35.923018 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:35.922534    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:35.947211 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:35.947162    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq" podStartSLOduration=2.451246788 podStartE2EDuration="6.947145358s" totalImagesPullingTime="3.432159943s" totalInitContainerRuntime="1.063738627s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:38:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:38:35.945770712 +0000 UTC m=+397.077059164" watchObservedRunningTime="2026-07-13 18:38:35.947145358 +0000 UTC m=+397.078433818"
Jul 13 18:38:46.928770 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:46.928738    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b4789cfbf-t92zq"
Jul 13 18:38:50.092082 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.092033    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hfcwh"]
Jul 13 18:38:50.095313 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.095298    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:38:50.098013 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.097986    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 13 18:38:50.098098 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.098055    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-8xjcz\""
Jul 13 18:38:50.110262 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.110237    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hfcwh"]
Jul 13 18:38:50.231760 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.231724    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjs57\" (UniqueName: \"kubernetes.io/projected/80269e47-5ba4-4b0a-b925-06cbf69049bd-kube-api-access-bjs57\") pod \"odh-model-controller-7998c67dc4-hfcwh\" (UID: \"80269e47-5ba4-4b0a-b925-06cbf69049bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:38:50.231968 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.231777    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80269e47-5ba4-4b0a-b925-06cbf69049bd-cert\") pod \"odh-model-controller-7998c67dc4-hfcwh\" (UID: \"80269e47-5ba4-4b0a-b925-06cbf69049bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:38:50.332450 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.332409    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjs57\" (UniqueName: \"kubernetes.io/projected/80269e47-5ba4-4b0a-b925-06cbf69049bd-kube-api-access-bjs57\") pod \"odh-model-controller-7998c67dc4-hfcwh\" (UID: \"80269e47-5ba4-4b0a-b925-06cbf69049bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:38:50.332624 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.332472    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80269e47-5ba4-4b0a-b925-06cbf69049bd-cert\") pod \"odh-model-controller-7998c67dc4-hfcwh\" (UID: \"80269e47-5ba4-4b0a-b925-06cbf69049bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:38:50.332705 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:38:50.332680    2569 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 13 18:38:50.332784 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:38:50.332771    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80269e47-5ba4-4b0a-b925-06cbf69049bd-cert podName:80269e47-5ba4-4b0a-b925-06cbf69049bd nodeName:}" failed. No retries permitted until 2026-07-13 18:38:50.832750364 +0000 UTC m=+411.964038818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/80269e47-5ba4-4b0a-b925-06cbf69049bd-cert") pod "odh-model-controller-7998c67dc4-hfcwh" (UID: "80269e47-5ba4-4b0a-b925-06cbf69049bd") : secret "odh-model-controller-webhook-cert" not found
Jul 13 18:38:50.369169 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.369099    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjs57\" (UniqueName: \"kubernetes.io/projected/80269e47-5ba4-4b0a-b925-06cbf69049bd-kube-api-access-bjs57\") pod \"odh-model-controller-7998c67dc4-hfcwh\" (UID: \"80269e47-5ba4-4b0a-b925-06cbf69049bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:38:50.836215 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.836176    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80269e47-5ba4-4b0a-b925-06cbf69049bd-cert\") pod \"odh-model-controller-7998c67dc4-hfcwh\" (UID: \"80269e47-5ba4-4b0a-b925-06cbf69049bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:38:50.838976 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:50.838948    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80269e47-5ba4-4b0a-b925-06cbf69049bd-cert\") pod \"odh-model-controller-7998c67dc4-hfcwh\" (UID: \"80269e47-5ba4-4b0a-b925-06cbf69049bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:38:51.006414 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.006377    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rcfd2"]
Jul 13 18:38:51.007912 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.007892    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:38:51.010277 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.010254    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2"
Jul 13 18:38:51.012625 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.012604    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 13 18:38:51.022992 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.022969    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rcfd2"]
Jul 13 18:38:51.138608 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.138577    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb9b384f-4d45-4297-8e1c-94d06747ed00-tls-certs\") pod \"model-serving-api-54d89bd5fc-rcfd2\" (UID: \"bb9b384f-4d45-4297-8e1c-94d06747ed00\") " pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2"
Jul 13 18:38:51.138992 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.138620    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr4gq\" (UniqueName: \"kubernetes.io/projected/bb9b384f-4d45-4297-8e1c-94d06747ed00-kube-api-access-qr4gq\") pod \"model-serving-api-54d89bd5fc-rcfd2\" (UID: \"bb9b384f-4d45-4297-8e1c-94d06747ed00\") " pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2"
Jul 13 18:38:51.156097 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.156063    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hfcwh"]
Jul 13 18:38:51.159238 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:38:51.159212    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80269e47_5ba4_4b0a_b925_06cbf69049bd.slice/crio-64b9ad50aee8216ab1a1e65461e34f16b5a59c10f97bf4e41b072ebacfa4b011 WatchSource:0}: Error finding container 64b9ad50aee8216ab1a1e65461e34f16b5a59c10f97bf4e41b072ebacfa4b011: Status 404 returned error can't find the container with id 64b9ad50aee8216ab1a1e65461e34f16b5a59c10f97bf4e41b072ebacfa4b011
Jul 13 18:38:51.239458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.239419    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb9b384f-4d45-4297-8e1c-94d06747ed00-tls-certs\") pod \"model-serving-api-54d89bd5fc-rcfd2\" (UID: \"bb9b384f-4d45-4297-8e1c-94d06747ed00\") " pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2"
Jul 13 18:38:51.239458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.239457    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr4gq\" (UniqueName: \"kubernetes.io/projected/bb9b384f-4d45-4297-8e1c-94d06747ed00-kube-api-access-qr4gq\") pod \"model-serving-api-54d89bd5fc-rcfd2\" (UID: \"bb9b384f-4d45-4297-8e1c-94d06747ed00\") " pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2"
Jul 13 18:38:51.242478 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.242457    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb9b384f-4d45-4297-8e1c-94d06747ed00-tls-certs\") pod \"model-serving-api-54d89bd5fc-rcfd2\" (UID: \"bb9b384f-4d45-4297-8e1c-94d06747ed00\") " pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2"
Jul 13 18:38:51.248545 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.248519    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr4gq\" (UniqueName: \"kubernetes.io/projected/bb9b384f-4d45-4297-8e1c-94d06747ed00-kube-api-access-qr4gq\") pod \"model-serving-api-54d89bd5fc-rcfd2\" (UID: \"bb9b384f-4d45-4297-8e1c-94d06747ed00\") " pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2"
Jul 13 18:38:51.342351 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.342315    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2"
Jul 13 18:38:51.484800 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.484777    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rcfd2"]
Jul 13 18:38:51.486874 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:38:51.486847    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb9b384f_4d45_4297_8e1c_94d06747ed00.slice/crio-bb6b5b04c0f41c052e51fd309cbde2d018c4bc215a973b04ee49981db79a0618 WatchSource:0}: Error finding container bb6b5b04c0f41c052e51fd309cbde2d018c4bc215a973b04ee49981db79a0618: Status 404 returned error can't find the container with id bb6b5b04c0f41c052e51fd309cbde2d018c4bc215a973b04ee49981db79a0618
Jul 13 18:38:51.989626 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.989591    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh" event={"ID":"80269e47-5ba4-4b0a-b925-06cbf69049bd","Type":"ContainerStarted","Data":"64b9ad50aee8216ab1a1e65461e34f16b5a59c10f97bf4e41b072ebacfa4b011"}
Jul 13 18:38:51.991039 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:51.991007    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2" event={"ID":"bb9b384f-4d45-4297-8e1c-94d06747ed00","Type":"ContainerStarted","Data":"bb6b5b04c0f41c052e51fd309cbde2d018c4bc215a973b04ee49981db79a0618"}
Jul 13 18:38:55.009893 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:55.009779    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2" event={"ID":"bb9b384f-4d45-4297-8e1c-94d06747ed00","Type":"ContainerStarted","Data":"9e14ad7ae46b108d5118e038f85e0726ca883d65d5ccd3bd8af57b588a4eca42"}
Jul 13 18:38:55.010362 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:55.009910    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2"
Jul 13 18:38:55.011401 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:55.011376    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh" event={"ID":"80269e47-5ba4-4b0a-b925-06cbf69049bd","Type":"ContainerStarted","Data":"458aaf38e88c4f235d8a94699b9efb88d7bdd177beb40f3894ed9c09060baea1"}
Jul 13 18:38:55.011521 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:55.011473    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:38:55.026629 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:55.026241    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2" podStartSLOduration=1.770711519 podStartE2EDuration="5.026227226s" totalImagesPullingTime="3.255515707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:38:55.026139777 +0000 UTC m=+416.157428238" watchObservedRunningTime="2026-07-13 18:38:55.026227226 +0000 UTC m=+416.157515686"
Jul 13 18:38:55.042289 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:38:55.042233    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh" podStartSLOduration=1.447837361 podStartE2EDuration="5.042210641s" totalImagesPullingTime="3.59437328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:38:55.04093223 +0000 UTC m=+416.172220692" watchObservedRunningTime="2026-07-13 18:38:55.042210641 +0000 UTC m=+416.173499103"
Jul 13 18:39:00.182283 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.182245    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66f8768797-p9r57"]
Jul 13 18:39:00.187111 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.187090    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.196439 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.196418    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 18:39:00.198483 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.198465    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 13 18:39:00.227556 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.227527    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66f8768797-p9r57"]
Jul 13 18:39:00.319045 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.319007    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0ab603f-cc81-4023-b68f-75c3bbdcaed7-tls-certs\") pod \"kube-auth-proxy-66f8768797-p9r57\" (UID: \"f0ab603f-cc81-4023-b68f-75c3bbdcaed7\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.319045 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.319047    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg6mw\" (UniqueName: \"kubernetes.io/projected/f0ab603f-cc81-4023-b68f-75c3bbdcaed7-kube-api-access-xg6mw\") pod \"kube-auth-proxy-66f8768797-p9r57\" (UID: \"f0ab603f-cc81-4023-b68f-75c3bbdcaed7\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.319249 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.319071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0ab603f-cc81-4023-b68f-75c3bbdcaed7-tmp\") pod \"kube-auth-proxy-66f8768797-p9r57\" (UID: \"f0ab603f-cc81-4023-b68f-75c3bbdcaed7\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.420068 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.420040    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0ab603f-cc81-4023-b68f-75c3bbdcaed7-tls-certs\") pod \"kube-auth-proxy-66f8768797-p9r57\" (UID: \"f0ab603f-cc81-4023-b68f-75c3bbdcaed7\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.420261 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.420081    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg6mw\" (UniqueName: \"kubernetes.io/projected/f0ab603f-cc81-4023-b68f-75c3bbdcaed7-kube-api-access-xg6mw\") pod \"kube-auth-proxy-66f8768797-p9r57\" (UID: \"f0ab603f-cc81-4023-b68f-75c3bbdcaed7\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.420261 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.420119    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0ab603f-cc81-4023-b68f-75c3bbdcaed7-tmp\") pod \"kube-auth-proxy-66f8768797-p9r57\" (UID: \"f0ab603f-cc81-4023-b68f-75c3bbdcaed7\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.422775 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.422747    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0ab603f-cc81-4023-b68f-75c3bbdcaed7-tmp\") pod \"kube-auth-proxy-66f8768797-p9r57\" (UID: \"f0ab603f-cc81-4023-b68f-75c3bbdcaed7\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.422991 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.422968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0ab603f-cc81-4023-b68f-75c3bbdcaed7-tls-certs\") pod \"kube-auth-proxy-66f8768797-p9r57\" (UID: \"f0ab603f-cc81-4023-b68f-75c3bbdcaed7\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.439889 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.439823    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg6mw\" (UniqueName: \"kubernetes.io/projected/f0ab603f-cc81-4023-b68f-75c3bbdcaed7-kube-api-access-xg6mw\") pod \"kube-auth-proxy-66f8768797-p9r57\" (UID: \"f0ab603f-cc81-4023-b68f-75c3bbdcaed7\") " pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.496040 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.496008    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57"
Jul 13 18:39:00.588369 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.588340    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"]
Jul 13 18:39:00.592294 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.592273    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:39:00.600705 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.600684    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 13 18:39:00.601005 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.600988    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vl2qm\""
Jul 13 18:39:00.608274 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.608249    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"]
Jul 13 18:39:00.619705 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.619681    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"]
Jul 13 18:39:00.623411 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.623393    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"
Jul 13 18:39:00.627060 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.627036    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-n4smk\""
Jul 13 18:39:00.627164 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.627136    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 13 18:39:00.638504 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.638481    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"]
Jul 13 18:39:00.644763 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.644742    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66f8768797-p9r57"]
Jul 13 18:39:00.648456 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:39:00.648434    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ab603f_cc81_4023_b68f_75c3bbdcaed7.slice/crio-2fb97209a2c727f0962e0e536433fe4e90d12868da40896cb1c4f9bc45672a46 WatchSource:0}: Error finding container 2fb97209a2c727f0962e0e536433fe4e90d12868da40896cb1c4f9bc45672a46: Status 404 returned error can't find the container with id 2fb97209a2c727f0962e0e536433fe4e90d12868da40896cb1c4f9bc45672a46
Jul 13 18:39:00.722788 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.722705    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhskq\" (UniqueName: \"kubernetes.io/projected/3693db18-1d4a-4c46-8cb9-75ba5c992a83-kube-api-access-vhskq\") pod \"kserve-controller-manager-d6f4b65b8-4wqcl\" (UID: \"3693db18-1d4a-4c46-8cb9-75ba5c992a83\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:39:00.722788 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.722776    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6p7j\" (UniqueName: \"kubernetes.io/projected/5ce2c5df-34db-4e6f-861e-a080808a2530-kube-api-access-x6p7j\") pod \"llmisvc-controller-manager-777c6bd7d5-zls45\" (UID: \"5ce2c5df-34db-4e6f-861e-a080808a2530\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"
Jul 13 18:39:00.723001 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.722843    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3693db18-1d4a-4c46-8cb9-75ba5c992a83-cert\") pod \"kserve-controller-manager-d6f4b65b8-4wqcl\" (UID: \"3693db18-1d4a-4c46-8cb9-75ba5c992a83\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:39:00.723001 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.722868    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ce2c5df-34db-4e6f-861e-a080808a2530-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zls45\" (UID: \"5ce2c5df-34db-4e6f-861e-a080808a2530\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"
Jul 13 18:39:00.824244 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.824202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6p7j\" (UniqueName: \"kubernetes.io/projected/5ce2c5df-34db-4e6f-861e-a080808a2530-kube-api-access-x6p7j\") pod \"llmisvc-controller-manager-777c6bd7d5-zls45\" (UID: \"5ce2c5df-34db-4e6f-861e-a080808a2530\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"
Jul 13 18:39:00.824533 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.824258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3693db18-1d4a-4c46-8cb9-75ba5c992a83-cert\") pod \"kserve-controller-manager-d6f4b65b8-4wqcl\" (UID: \"3693db18-1d4a-4c46-8cb9-75ba5c992a83\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:39:00.824533 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.824280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ce2c5df-34db-4e6f-861e-a080808a2530-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zls45\" (UID: \"5ce2c5df-34db-4e6f-861e-a080808a2530\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"
Jul 13 18:39:00.824533 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.824333    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhskq\" (UniqueName: \"kubernetes.io/projected/3693db18-1d4a-4c46-8cb9-75ba5c992a83-kube-api-access-vhskq\") pod \"kserve-controller-manager-d6f4b65b8-4wqcl\" (UID: \"3693db18-1d4a-4c46-8cb9-75ba5c992a83\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:39:00.824533 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:39:00.824413    2569 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 13 18:39:00.824533 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:39:00.824466    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3693db18-1d4a-4c46-8cb9-75ba5c992a83-cert podName:3693db18-1d4a-4c46-8cb9-75ba5c992a83 nodeName:}" failed. No retries permitted until 2026-07-13 18:39:01.324452036 +0000 UTC m=+422.455740474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3693db18-1d4a-4c46-8cb9-75ba5c992a83-cert") pod "kserve-controller-manager-d6f4b65b8-4wqcl" (UID: "3693db18-1d4a-4c46-8cb9-75ba5c992a83") : secret "kserve-webhook-server-cert" not found
Jul 13 18:39:00.826808 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.826790    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ce2c5df-34db-4e6f-861e-a080808a2530-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zls45\" (UID: \"5ce2c5df-34db-4e6f-861e-a080808a2530\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"
Jul 13 18:39:00.838072 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.838040    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhskq\" (UniqueName: \"kubernetes.io/projected/3693db18-1d4a-4c46-8cb9-75ba5c992a83-kube-api-access-vhskq\") pod \"kserve-controller-manager-d6f4b65b8-4wqcl\" (UID: \"3693db18-1d4a-4c46-8cb9-75ba5c992a83\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:39:00.843820 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.843804    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6p7j\" (UniqueName: \"kubernetes.io/projected/5ce2c5df-34db-4e6f-861e-a080808a2530-kube-api-access-x6p7j\") pod \"llmisvc-controller-manager-777c6bd7d5-zls45\" (UID: \"5ce2c5df-34db-4e6f-861e-a080808a2530\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"
Jul 13 18:39:00.933414 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:00.933364    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"
Jul 13 18:39:01.040723 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:01.040690    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57" event={"ID":"f0ab603f-cc81-4023-b68f-75c3bbdcaed7","Type":"ContainerStarted","Data":"2fb97209a2c727f0962e0e536433fe4e90d12868da40896cb1c4f9bc45672a46"}
Jul 13 18:39:01.116392 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:01.116285    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"]
Jul 13 18:39:01.118817 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:39:01.118787    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5ce2c5df_34db_4e6f_861e_a080808a2530.slice/crio-994124b9d4ecc2151181454dfc3b6494801b9a24a750eee60f11b85a19289699 WatchSource:0}: Error finding container 994124b9d4ecc2151181454dfc3b6494801b9a24a750eee60f11b85a19289699: Status 404 returned error can't find the container with id 994124b9d4ecc2151181454dfc3b6494801b9a24a750eee60f11b85a19289699
Jul 13 18:39:01.329615 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:01.329529    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3693db18-1d4a-4c46-8cb9-75ba5c992a83-cert\") pod \"kserve-controller-manager-d6f4b65b8-4wqcl\" (UID: \"3693db18-1d4a-4c46-8cb9-75ba5c992a83\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:39:01.332017 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:01.331996    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3693db18-1d4a-4c46-8cb9-75ba5c992a83-cert\") pod \"kserve-controller-manager-d6f4b65b8-4wqcl\" (UID: \"3693db18-1d4a-4c46-8cb9-75ba5c992a83\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:39:01.505974 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:01.505942    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:39:01.677372 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:01.677337    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"]
Jul 13 18:39:01.680595 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:39:01.680555    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3693db18_1d4a_4c46_8cb9_75ba5c992a83.slice/crio-05c6e8aa2ac3cbe5cc2562082c21e00a5b4b0596c4c15e7df2d1a6fc4960abe4 WatchSource:0}: Error finding container 05c6e8aa2ac3cbe5cc2562082c21e00a5b4b0596c4c15e7df2d1a6fc4960abe4: Status 404 returned error can't find the container with id 05c6e8aa2ac3cbe5cc2562082c21e00a5b4b0596c4c15e7df2d1a6fc4960abe4
Jul 13 18:39:02.048573 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:02.048542    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl" event={"ID":"3693db18-1d4a-4c46-8cb9-75ba5c992a83","Type":"ContainerStarted","Data":"05c6e8aa2ac3cbe5cc2562082c21e00a5b4b0596c4c15e7df2d1a6fc4960abe4"}
Jul 13 18:39:02.049777 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:02.049753    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45" event={"ID":"5ce2c5df-34db-4e6f-861e-a080808a2530","Type":"ContainerStarted","Data":"994124b9d4ecc2151181454dfc3b6494801b9a24a750eee60f11b85a19289699"}
Jul 13 18:39:03.376176 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.376139    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"]
Jul 13 18:39:03.388490 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.388461    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.400542 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.400515    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vp6x7\""
Jul 13 18:39:03.401188 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.401167    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 13 18:39:03.401188 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.401179    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 13 18:39:03.401349 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.401295    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 13 18:39:03.402130 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.402115    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 13 18:39:03.428011 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.427977    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"]
Jul 13 18:39:03.448740 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.448713    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tl7j\" (UniqueName: \"kubernetes.io/projected/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-kube-api-access-2tl7j\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.448898 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.448764    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.448898 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.448790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.449023 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.448951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.449075 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.449029    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.449075 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.449059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.449188 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.449084    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.550372 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.550318    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tl7j\" (UniqueName: \"kubernetes.io/projected/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-kube-api-access-2tl7j\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.550571 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.550378    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.550571 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.550406    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.550571 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.550487    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.550571 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.550546    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.550571 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.550561    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.550881 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.550580    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.551339 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.551308    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.553625 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.553603    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.553771 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.553711    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.553845 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.553778    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.554384 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.554359    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.561944 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.561920    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tl7j\" (UniqueName: \"kubernetes.io/projected/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-kube-api-access-2tl7j\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.563644 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.563604    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba045cfc-ce28-452c-9bfc-f9d30a84d1fb-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6jt48\" (UID: \"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:03.699826 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:03.699739    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:04.781573 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:04.781538    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"]
Jul 13 18:39:04.797953 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:39:04.797925    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba045cfc_ce28_452c_9bfc_f9d30a84d1fb.slice/crio-4df6e539ab4c482f8fede0c2efb3c68f76d196be0fba4bc062d6e152f3e2c172 WatchSource:0}: Error finding container 4df6e539ab4c482f8fede0c2efb3c68f76d196be0fba4bc062d6e152f3e2c172: Status 404 returned error can't find the container with id 4df6e539ab4c482f8fede0c2efb3c68f76d196be0fba4bc062d6e152f3e2c172
Jul 13 18:39:05.071397 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:05.071363    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45" event={"ID":"5ce2c5df-34db-4e6f-861e-a080808a2530","Type":"ContainerStarted","Data":"201830b0ccdc03c571a981331822867094d5848560902d475b7325b312cf8767"}
Jul 13 18:39:05.071602 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:05.071482    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"
Jul 13 18:39:05.072669 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:05.072620    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48" event={"ID":"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb","Type":"ContainerStarted","Data":"4df6e539ab4c482f8fede0c2efb3c68f76d196be0fba4bc062d6e152f3e2c172"}
Jul 13 18:39:05.073997 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:05.073965    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57" event={"ID":"f0ab603f-cc81-4023-b68f-75c3bbdcaed7","Type":"ContainerStarted","Data":"e08f4a290cfd1bd697e6bc4c751b0ee0574ba8be3bc8c957e8f968da2a1fc3bb"}
Jul 13 18:39:05.092492 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:05.092440    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45" podStartSLOduration=1.60994614 podStartE2EDuration="5.092427199s" totalImagesPullingTime="3.482481059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:39:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:39:05.091463075 +0000 UTC m=+426.222751534" watchObservedRunningTime="2026-07-13 18:39:05.092427199 +0000 UTC m=+426.223715659"
Jul 13 18:39:05.115566 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:05.115503    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66f8768797-p9r57" podStartSLOduration=1.168425478 podStartE2EDuration="5.1154813s" totalImagesPullingTime="3.947055822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:39:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:39:05.113602712 +0000 UTC m=+426.244891173" watchObservedRunningTime="2026-07-13 18:39:05.1154813 +0000 UTC m=+426.246769761"
Jul 13 18:39:06.018279 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:06.018244    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hfcwh"
Jul 13 18:39:06.020909 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:06.020883    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-rcfd2"
Jul 13 18:39:07.085580 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:07.085530    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl" event={"ID":"3693db18-1d4a-4c46-8cb9-75ba5c992a83","Type":"ContainerStarted","Data":"41955a76ba12217ea6cc5fd651c7da3b457a6c9c4fcfe03256ce5ad0bcddde17"}
Jul 13 18:39:07.086019 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:07.085689    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:39:07.110318 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:07.110267    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl" podStartSLOduration=2.574153142 podStartE2EDuration="7.110251828s" totalImagesPullingTime="4.536098686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:39:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:39:07.108491207 +0000 UTC m=+428.239779667" watchObservedRunningTime="2026-07-13 18:39:07.110251828 +0000 UTC m=+428.241540288"
Jul 13 18:39:08.434962 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:08.434232    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 18:39:08.434962 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:08.434314    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 18:39:09.097078 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:09.097041    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48" event={"ID":"ba045cfc-ce28-452c-9bfc-f9d30a84d1fb","Type":"ContainerStarted","Data":"3716e0bb4bd88bf77282f59c2f9b16a2b43cdf942502646dad38c2f0608a9d9e"}
Jul 13 18:39:09.097264 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:09.097185    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:09.098703 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:09.098677    2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-6jt48 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 13 18:39:09.098834 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:09.098729    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48" podUID="ba045cfc-ce28-452c-9bfc-f9d30a84d1fb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 18:39:09.119071 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:09.119020    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48" podStartSLOduration=2.485511282 podStartE2EDuration="6.119002277s" totalImagesPullingTime="3.633490995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:39:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:39:09.118162249 +0000 UTC m=+430.249450708" watchObservedRunningTime="2026-07-13 18:39:09.119002277 +0000 UTC m=+430.250290740"
Jul 13 18:39:10.102446 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:10.102420    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6jt48"
Jul 13 18:39:36.082172 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:36.082094    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zls45"
Jul 13 18:39:38.095919 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:39:38.095890    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wqcl"
Jul 13 18:40:02.518544 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.518504    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"]
Jul 13 18:40:02.521907 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.521885    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.524324 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.524306    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 18:40:02.524419 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.524308    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kkqv9\""
Jul 13 18:40:02.525050 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.525037    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 18:40:02.531367 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.531347    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"]
Jul 13 18:40:02.643233 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.643204    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.643439 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.643251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w692s\" (UniqueName: \"kubernetes.io/projected/7da49cce-9637-428c-a819-f878fb853e3e-kube-api-access-w692s\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.643439 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.643270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.744673 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.744628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.744851 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.744721    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w692s\" (UniqueName: \"kubernetes.io/projected/7da49cce-9637-428c-a819-f878fb853e3e-kube-api-access-w692s\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.744851 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.744750    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.744973 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.744956    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.745120 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.745102    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.752854 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.752829    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w692s\" (UniqueName: \"kubernetes.io/projected/7da49cce-9637-428c-a819-f878fb853e3e-kube-api-access-w692s\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.831896 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.831801    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:02.966407 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:02.966382    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"]
Jul 13 18:40:02.968557 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:40:02.968528    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da49cce_9637_428c_a819_f878fb853e3e.slice/crio-e4083b9774e995767160af4193b18a578c36cc17c1789196169acdbf4a6187d5 WatchSource:0}: Error finding container e4083b9774e995767160af4193b18a578c36cc17c1789196169acdbf4a6187d5: Status 404 returned error can't find the container with id e4083b9774e995767160af4193b18a578c36cc17c1789196169acdbf4a6187d5
Jul 13 18:40:03.112464 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.112403    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"]
Jul 13 18:40:03.115607 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.115585    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.124706 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.124682    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"]
Jul 13 18:40:03.250079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.250038    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.250079 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.250081    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7wwd\" (UniqueName: \"kubernetes.io/projected/83168f79-3b2b-402e-8f90-6c712e45c39d-kube-api-access-q7wwd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.250280 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.250141    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.344177 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.344147    2569 generic.go:356] "Generic (PLEG): container finished" podID="7da49cce-9637-428c-a819-f878fb853e3e" containerID="99d87747798c1a42c9e9973570f5e4576fea2e5e433514dad47d0e355dff2e7a" exitCode=0
Jul 13 18:40:03.348966 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.348940    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r" event={"ID":"7da49cce-9637-428c-a819-f878fb853e3e","Type":"ContainerDied","Data":"99d87747798c1a42c9e9973570f5e4576fea2e5e433514dad47d0e355dff2e7a"}
Jul 13 18:40:03.349047 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.348985    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r" event={"ID":"7da49cce-9637-428c-a819-f878fb853e3e","Type":"ContainerStarted","Data":"e4083b9774e995767160af4193b18a578c36cc17c1789196169acdbf4a6187d5"}
Jul 13 18:40:03.350938 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.350916    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.351035 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.350953    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7wwd\" (UniqueName: \"kubernetes.io/projected/83168f79-3b2b-402e-8f90-6c712e45c39d-kube-api-access-q7wwd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.351035 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.350988    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.351313 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.351293    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.351377 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.351334    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.361772 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.361746    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7wwd\" (UniqueName: \"kubernetes.io/projected/83168f79-3b2b-402e-8f90-6c712e45c39d-kube-api-access-q7wwd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.448323 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.448250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:03.577761 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.577735    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"]
Jul 13 18:40:03.579478 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:40:03.579453    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83168f79_3b2b_402e_8f90_6c712e45c39d.slice/crio-5316b2d716e6eb27b6619d2043836ca59b3d2d2cc1ef75c5040fb33375df7fbd WatchSource:0}: Error finding container 5316b2d716e6eb27b6619d2043836ca59b3d2d2cc1ef75c5040fb33375df7fbd: Status 404 returned error can't find the container with id 5316b2d716e6eb27b6619d2043836ca59b3d2d2cc1ef75c5040fb33375df7fbd
Jul 13 18:40:03.706099 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.706009    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"]
Jul 13 18:40:03.709276 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.709260    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:03.717666 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.717630    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"]
Jul 13 18:40:03.855328 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.855295    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:03.855565 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.855370    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:03.855565 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.855410    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ngcv\" (UniqueName: \"kubernetes.io/projected/25a50f5b-09ad-48b7-9df1-08657541b55e-kube-api-access-4ngcv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:03.956683 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.956581    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:03.956683 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.956626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ngcv\" (UniqueName: \"kubernetes.io/projected/25a50f5b-09ad-48b7-9df1-08657541b55e-kube-api-access-4ngcv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:03.956901 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.956698    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:03.957001 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.956982    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:03.957064 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.957040    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:03.964942 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:03.964920    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ngcv\" (UniqueName: \"kubernetes.io/projected/25a50f5b-09ad-48b7-9df1-08657541b55e-kube-api-access-4ngcv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:04.066337 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.066303    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:04.196266 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.196238    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"]
Jul 13 18:40:04.198624 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:40:04.198596    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a50f5b_09ad_48b7_9df1_08657541b55e.slice/crio-bd0c0a64c8bf2869a312ced48d672ccba15164510859d7dacfc00863d4c5aed3 WatchSource:0}: Error finding container bd0c0a64c8bf2869a312ced48d672ccba15164510859d7dacfc00863d4c5aed3: Status 404 returned error can't find the container with id bd0c0a64c8bf2869a312ced48d672ccba15164510859d7dacfc00863d4c5aed3
Jul 13 18:40:04.314125 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.314100    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"]
Jul 13 18:40:04.317626 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.317609    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.329002 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.328978    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"]
Jul 13 18:40:04.350711 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.350676    2569 generic.go:356] "Generic (PLEG): container finished" podID="25a50f5b-09ad-48b7-9df1-08657541b55e" containerID="7a2fbbc25ba28c6127980329da9704b05ff8e7d5ad8e535a0ead06591554d447" exitCode=0
Jul 13 18:40:04.350857 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.350714    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp" event={"ID":"25a50f5b-09ad-48b7-9df1-08657541b55e","Type":"ContainerDied","Data":"7a2fbbc25ba28c6127980329da9704b05ff8e7d5ad8e535a0ead06591554d447"}
Jul 13 18:40:04.350857 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.350761    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp" event={"ID":"25a50f5b-09ad-48b7-9df1-08657541b55e","Type":"ContainerStarted","Data":"bd0c0a64c8bf2869a312ced48d672ccba15164510859d7dacfc00863d4c5aed3"}
Jul 13 18:40:04.352703 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.352678    2569 generic.go:356] "Generic (PLEG): container finished" podID="83168f79-3b2b-402e-8f90-6c712e45c39d" containerID="ae5d1909c4abe1efadc03171c010021b00d67ea9d7031c9ffe91dff218abdaba" exitCode=0
Jul 13 18:40:04.352825 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.352759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x" event={"ID":"83168f79-3b2b-402e-8f90-6c712e45c39d","Type":"ContainerDied","Data":"ae5d1909c4abe1efadc03171c010021b00d67ea9d7031c9ffe91dff218abdaba"}
Jul 13 18:40:04.352825 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.352808    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x" event={"ID":"83168f79-3b2b-402e-8f90-6c712e45c39d","Type":"ContainerStarted","Data":"5316b2d716e6eb27b6619d2043836ca59b3d2d2cc1ef75c5040fb33375df7fbd"}
Jul 13 18:40:04.460062 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.460018    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.460249 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.460150    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr4mt\" (UniqueName: \"kubernetes.io/projected/bfef29e3-76c2-4926-9db0-764c65cc63cd-kube-api-access-sr4mt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.460249 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.460224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.561544 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.561522    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.561607 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.561587    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr4mt\" (UniqueName: \"kubernetes.io/projected/bfef29e3-76c2-4926-9db0-764c65cc63cd-kube-api-access-sr4mt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.561671 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.561630    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.561869 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.561849    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.561948 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.561895    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.569441 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.569421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr4mt\" (UniqueName: \"kubernetes.io/projected/bfef29e3-76c2-4926-9db0-764c65cc63cd-kube-api-access-sr4mt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.662603 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.662574    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:04.794246 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:04.794215    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"]
Jul 13 18:40:04.796533 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:40:04.796468    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfef29e3_76c2_4926_9db0_764c65cc63cd.slice/crio-68a9eef8d86eae69644bf1f74c0e7404a88460ee51cfe32c12540e39df4f7cb7 WatchSource:0}: Error finding container 68a9eef8d86eae69644bf1f74c0e7404a88460ee51cfe32c12540e39df4f7cb7: Status 404 returned error can't find the container with id 68a9eef8d86eae69644bf1f74c0e7404a88460ee51cfe32c12540e39df4f7cb7
Jul 13 18:40:05.357919 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:05.357893    2569 generic.go:356] "Generic (PLEG): container finished" podID="7da49cce-9637-428c-a819-f878fb853e3e" containerID="01be416dbaff8ac81288a08552bdce9e69e7eb51253e5b63a15c7c3c8c00972c" exitCode=0
Jul 13 18:40:05.358019 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:05.357980    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r" event={"ID":"7da49cce-9637-428c-a819-f878fb853e3e","Type":"ContainerDied","Data":"01be416dbaff8ac81288a08552bdce9e69e7eb51253e5b63a15c7c3c8c00972c"}
Jul 13 18:40:05.362602 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:05.362582    2569 generic.go:356] "Generic (PLEG): container finished" podID="bfef29e3-76c2-4926-9db0-764c65cc63cd" containerID="e78bb10bc9a868bf9664a764d24f6f3d5ca801885151f0af1f46000ad040a3b9" exitCode=0
Jul 13 18:40:05.362717 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:05.362614    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z" event={"ID":"bfef29e3-76c2-4926-9db0-764c65cc63cd","Type":"ContainerDied","Data":"e78bb10bc9a868bf9664a764d24f6f3d5ca801885151f0af1f46000ad040a3b9"}
Jul 13 18:40:05.362717 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:05.362670    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z" event={"ID":"bfef29e3-76c2-4926-9db0-764c65cc63cd","Type":"ContainerStarted","Data":"68a9eef8d86eae69644bf1f74c0e7404a88460ee51cfe32c12540e39df4f7cb7"}
Jul 13 18:40:06.369399 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:06.369353    2569 generic.go:356] "Generic (PLEG): container finished" podID="25a50f5b-09ad-48b7-9df1-08657541b55e" containerID="81feb12c0f4a7f27d001ef3833aa468ea1435d249e1cf6ead95ed316f10a90d4" exitCode=0
Jul 13 18:40:06.369850 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:06.369422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp" event={"ID":"25a50f5b-09ad-48b7-9df1-08657541b55e","Type":"ContainerDied","Data":"81feb12c0f4a7f27d001ef3833aa468ea1435d249e1cf6ead95ed316f10a90d4"}
Jul 13 18:40:06.371684 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:06.371641    2569 generic.go:356] "Generic (PLEG): container finished" podID="7da49cce-9637-428c-a819-f878fb853e3e" containerID="398ef721c4e6276bca76b56969d2ee0872f2c21ef51f59da5c221e87552e9a5b" exitCode=0
Jul 13 18:40:06.371786 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:06.371688    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r" event={"ID":"7da49cce-9637-428c-a819-f878fb853e3e","Type":"ContainerDied","Data":"398ef721c4e6276bca76b56969d2ee0872f2c21ef51f59da5c221e87552e9a5b"}
Jul 13 18:40:06.373379 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:06.373350    2569 generic.go:356] "Generic (PLEG): container finished" podID="83168f79-3b2b-402e-8f90-6c712e45c39d" containerID="7885aa5c3f80e098db8efa49c8df45a017f6cc17856b0908efef048446462828" exitCode=0
Jul 13 18:40:06.373467 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:06.373380    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x" event={"ID":"83168f79-3b2b-402e-8f90-6c712e45c39d","Type":"ContainerDied","Data":"7885aa5c3f80e098db8efa49c8df45a017f6cc17856b0908efef048446462828"}
Jul 13 18:40:07.380232 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.380140    2569 generic.go:356] "Generic (PLEG): container finished" podID="83168f79-3b2b-402e-8f90-6c712e45c39d" containerID="c366d1c9ab058f951d9dff564ec0d67d8f63246e39df6cbd7eb52d4bf08b9783" exitCode=0
Jul 13 18:40:07.380232 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.380194    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x" event={"ID":"83168f79-3b2b-402e-8f90-6c712e45c39d","Type":"ContainerDied","Data":"c366d1c9ab058f951d9dff564ec0d67d8f63246e39df6cbd7eb52d4bf08b9783"}
Jul 13 18:40:07.381988 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.381963    2569 generic.go:356] "Generic (PLEG): container finished" podID="bfef29e3-76c2-4926-9db0-764c65cc63cd" containerID="b583f5c189d2e3e1ad1aa761b83cfab40246b98e7c656ccd7f198a82c0dae398" exitCode=0
Jul 13 18:40:07.381988 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.381987    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z" event={"ID":"bfef29e3-76c2-4926-9db0-764c65cc63cd","Type":"ContainerDied","Data":"b583f5c189d2e3e1ad1aa761b83cfab40246b98e7c656ccd7f198a82c0dae398"}
Jul 13 18:40:07.384291 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.384269    2569 generic.go:356] "Generic (PLEG): container finished" podID="25a50f5b-09ad-48b7-9df1-08657541b55e" containerID="3861bbbff4fd09334471449fab467b7a46bda8fc1cf260cc66bad4388ef72dd3" exitCode=0
Jul 13 18:40:07.384410 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.384394    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp" event={"ID":"25a50f5b-09ad-48b7-9df1-08657541b55e","Type":"ContainerDied","Data":"3861bbbff4fd09334471449fab467b7a46bda8fc1cf260cc66bad4388ef72dd3"}
Jul 13 18:40:07.516913 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.516887    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:07.688723 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.688627    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w692s\" (UniqueName: \"kubernetes.io/projected/7da49cce-9637-428c-a819-f878fb853e3e-kube-api-access-w692s\") pod \"7da49cce-9637-428c-a819-f878fb853e3e\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") "
Jul 13 18:40:07.688723 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.688702    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-bundle\") pod \"7da49cce-9637-428c-a819-f878fb853e3e\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") "
Jul 13 18:40:07.688951 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.688814    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-util\") pod \"7da49cce-9637-428c-a819-f878fb853e3e\" (UID: \"7da49cce-9637-428c-a819-f878fb853e3e\") "
Jul 13 18:40:07.689323 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.689296    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-bundle" (OuterVolumeSpecName: "bundle") pod "7da49cce-9637-428c-a819-f878fb853e3e" (UID: "7da49cce-9637-428c-a819-f878fb853e3e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:40:07.690966 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.690946    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7da49cce-9637-428c-a819-f878fb853e3e-kube-api-access-w692s" (OuterVolumeSpecName: "kube-api-access-w692s") pod "7da49cce-9637-428c-a819-f878fb853e3e" (UID: "7da49cce-9637-428c-a819-f878fb853e3e"). InnerVolumeSpecName "kube-api-access-w692s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:40:07.694037 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.694004    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-util" (OuterVolumeSpecName: "util") pod "7da49cce-9637-428c-a819-f878fb853e3e" (UID: "7da49cce-9637-428c-a819-f878fb853e3e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:40:07.789936 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.789898    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-util\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:07.789936 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.789931    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w692s\" (UniqueName: \"kubernetes.io/projected/7da49cce-9637-428c-a819-f878fb853e3e-kube-api-access-w692s\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:07.789936 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:07.789941    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7da49cce-9637-428c-a819-f878fb853e3e-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:08.393343 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.393314    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r"
Jul 13 18:40:08.393828 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.393311    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r" event={"ID":"7da49cce-9637-428c-a819-f878fb853e3e","Type":"ContainerDied","Data":"e4083b9774e995767160af4193b18a578c36cc17c1789196169acdbf4a6187d5"}
Jul 13 18:40:08.393828 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.393458    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4083b9774e995767160af4193b18a578c36cc17c1789196169acdbf4a6187d5"
Jul 13 18:40:08.395394 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.395372    2569 generic.go:356] "Generic (PLEG): container finished" podID="bfef29e3-76c2-4926-9db0-764c65cc63cd" containerID="2f8b2faa15387ebf3aa2f1d157f4e1cb2cf7eee972b736a5611b831cd1d0d3f9" exitCode=0
Jul 13 18:40:08.395568 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.395446    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z" event={"ID":"bfef29e3-76c2-4926-9db0-764c65cc63cd","Type":"ContainerDied","Data":"2f8b2faa15387ebf3aa2f1d157f4e1cb2cf7eee972b736a5611b831cd1d0d3f9"}
Jul 13 18:40:08.550338 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.550319    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:08.552378 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.552361    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:08.698975 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.698894    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-util\") pod \"83168f79-3b2b-402e-8f90-6c712e45c39d\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") "
Jul 13 18:40:08.698975 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.698960    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-bundle\") pod \"25a50f5b-09ad-48b7-9df1-08657541b55e\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") "
Jul 13 18:40:08.699164 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.698987    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7wwd\" (UniqueName: \"kubernetes.io/projected/83168f79-3b2b-402e-8f90-6c712e45c39d-kube-api-access-q7wwd\") pod \"83168f79-3b2b-402e-8f90-6c712e45c39d\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") "
Jul 13 18:40:08.699164 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.699036    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ngcv\" (UniqueName: \"kubernetes.io/projected/25a50f5b-09ad-48b7-9df1-08657541b55e-kube-api-access-4ngcv\") pod \"25a50f5b-09ad-48b7-9df1-08657541b55e\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") "
Jul 13 18:40:08.699164 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.699063    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-bundle\") pod \"83168f79-3b2b-402e-8f90-6c712e45c39d\" (UID: \"83168f79-3b2b-402e-8f90-6c712e45c39d\") "
Jul 13 18:40:08.699307 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.699193    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-util\") pod \"25a50f5b-09ad-48b7-9df1-08657541b55e\" (UID: \"25a50f5b-09ad-48b7-9df1-08657541b55e\") "
Jul 13 18:40:08.699582 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.699545    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-bundle" (OuterVolumeSpecName: "bundle") pod "25a50f5b-09ad-48b7-9df1-08657541b55e" (UID: "25a50f5b-09ad-48b7-9df1-08657541b55e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:40:08.699582 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.699548    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-bundle" (OuterVolumeSpecName: "bundle") pod "83168f79-3b2b-402e-8f90-6c712e45c39d" (UID: "83168f79-3b2b-402e-8f90-6c712e45c39d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:40:08.701420 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.701397    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25a50f5b-09ad-48b7-9df1-08657541b55e-kube-api-access-4ngcv" (OuterVolumeSpecName: "kube-api-access-4ngcv") pod "25a50f5b-09ad-48b7-9df1-08657541b55e" (UID: "25a50f5b-09ad-48b7-9df1-08657541b55e"). InnerVolumeSpecName "kube-api-access-4ngcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:40:08.701512 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.701446    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83168f79-3b2b-402e-8f90-6c712e45c39d-kube-api-access-q7wwd" (OuterVolumeSpecName: "kube-api-access-q7wwd") pod "83168f79-3b2b-402e-8f90-6c712e45c39d" (UID: "83168f79-3b2b-402e-8f90-6c712e45c39d"). InnerVolumeSpecName "kube-api-access-q7wwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:40:08.704358 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.704331    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-util" (OuterVolumeSpecName: "util") pod "25a50f5b-09ad-48b7-9df1-08657541b55e" (UID: "25a50f5b-09ad-48b7-9df1-08657541b55e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:40:08.704921 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.704903    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-util" (OuterVolumeSpecName: "util") pod "83168f79-3b2b-402e-8f90-6c712e45c39d" (UID: "83168f79-3b2b-402e-8f90-6c712e45c39d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:40:08.800440 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.800410    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-util\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:08.800440 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.800437    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:08.800440 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.800446    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7wwd\" (UniqueName: \"kubernetes.io/projected/83168f79-3b2b-402e-8f90-6c712e45c39d-kube-api-access-q7wwd\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:08.800690 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.800454    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ngcv\" (UniqueName: \"kubernetes.io/projected/25a50f5b-09ad-48b7-9df1-08657541b55e-kube-api-access-4ngcv\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:08.800690 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.800464    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83168f79-3b2b-402e-8f90-6c712e45c39d-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:08.800690 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:08.800472    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25a50f5b-09ad-48b7-9df1-08657541b55e-util\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:09.403073 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.403043    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x"
Jul 13 18:40:09.403515 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.403072    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x" event={"ID":"83168f79-3b2b-402e-8f90-6c712e45c39d","Type":"ContainerDied","Data":"5316b2d716e6eb27b6619d2043836ca59b3d2d2cc1ef75c5040fb33375df7fbd"}
Jul 13 18:40:09.403515 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.403127    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5316b2d716e6eb27b6619d2043836ca59b3d2d2cc1ef75c5040fb33375df7fbd"
Jul 13 18:40:09.404901 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.404873    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp" event={"ID":"25a50f5b-09ad-48b7-9df1-08657541b55e","Type":"ContainerDied","Data":"bd0c0a64c8bf2869a312ced48d672ccba15164510859d7dacfc00863d4c5aed3"}
Jul 13 18:40:09.405025 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.404916    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd0c0a64c8bf2869a312ced48d672ccba15164510859d7dacfc00863d4c5aed3"
Jul 13 18:40:09.405083 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.405067    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp"
Jul 13 18:40:09.543259 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.543235    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:09.707787 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.707710    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-bundle\") pod \"bfef29e3-76c2-4926-9db0-764c65cc63cd\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") "
Jul 13 18:40:09.707787 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.707746    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr4mt\" (UniqueName: \"kubernetes.io/projected/bfef29e3-76c2-4926-9db0-764c65cc63cd-kube-api-access-sr4mt\") pod \"bfef29e3-76c2-4926-9db0-764c65cc63cd\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") "
Jul 13 18:40:09.707960 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.707812    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-util\") pod \"bfef29e3-76c2-4926-9db0-764c65cc63cd\" (UID: \"bfef29e3-76c2-4926-9db0-764c65cc63cd\") "
Jul 13 18:40:09.708443 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.708345    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-bundle" (OuterVolumeSpecName: "bundle") pod "bfef29e3-76c2-4926-9db0-764c65cc63cd" (UID: "bfef29e3-76c2-4926-9db0-764c65cc63cd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:40:09.709979 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.709955    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfef29e3-76c2-4926-9db0-764c65cc63cd-kube-api-access-sr4mt" (OuterVolumeSpecName: "kube-api-access-sr4mt") pod "bfef29e3-76c2-4926-9db0-764c65cc63cd" (UID: "bfef29e3-76c2-4926-9db0-764c65cc63cd"). InnerVolumeSpecName "kube-api-access-sr4mt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:40:09.716058 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.716026    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-util" (OuterVolumeSpecName: "util") pod "bfef29e3-76c2-4926-9db0-764c65cc63cd" (UID: "bfef29e3-76c2-4926-9db0-764c65cc63cd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 18:40:09.809118 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.809063    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:09.809118 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.809112    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr4mt\" (UniqueName: \"kubernetes.io/projected/bfef29e3-76c2-4926-9db0-764c65cc63cd-kube-api-access-sr4mt\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:09.809118 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:09.809124    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfef29e3-76c2-4926-9db0-764c65cc63cd-util\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:10.029532 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029493    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86dfbf7594-92rxg"]
Jul 13 18:40:10.029878 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029861    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da49cce-9637-428c-a819-f878fb853e3e" containerName="util"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029881    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da49cce-9637-428c-a819-f878fb853e3e" containerName="util"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029896    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da49cce-9637-428c-a819-f878fb853e3e" containerName="pull"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029903    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da49cce-9637-428c-a819-f878fb853e3e" containerName="pull"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029914    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a50f5b-09ad-48b7-9df1-08657541b55e" containerName="extract"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029922    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a50f5b-09ad-48b7-9df1-08657541b55e" containerName="extract"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029932    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83168f79-3b2b-402e-8f90-6c712e45c39d" containerName="extract"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029940    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="83168f79-3b2b-402e-8f90-6c712e45c39d" containerName="extract"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029948    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfef29e3-76c2-4926-9db0-764c65cc63cd" containerName="extract"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029957    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfef29e3-76c2-4926-9db0-764c65cc63cd" containerName="extract"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029969    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83168f79-3b2b-402e-8f90-6c712e45c39d" containerName="util"
Jul 13 18:40:10.029978 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029977    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="83168f79-3b2b-402e-8f90-6c712e45c39d" containerName="util"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029988    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a50f5b-09ad-48b7-9df1-08657541b55e" containerName="util"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.029996    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a50f5b-09ad-48b7-9df1-08657541b55e" containerName="util"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030005    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfef29e3-76c2-4926-9db0-764c65cc63cd" containerName="util"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030012    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfef29e3-76c2-4926-9db0-764c65cc63cd" containerName="util"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030020    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a50f5b-09ad-48b7-9df1-08657541b55e" containerName="pull"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030027    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a50f5b-09ad-48b7-9df1-08657541b55e" containerName="pull"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030037    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da49cce-9637-428c-a819-f878fb853e3e" containerName="extract"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030046    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da49cce-9637-428c-a819-f878fb853e3e" containerName="extract"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030054    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfef29e3-76c2-4926-9db0-764c65cc63cd" containerName="pull"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030061    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfef29e3-76c2-4926-9db0-764c65cc63cd" containerName="pull"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030070    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83168f79-3b2b-402e-8f90-6c712e45c39d" containerName="pull"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030077    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="83168f79-3b2b-402e-8f90-6c712e45c39d" containerName="pull"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030165    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="25a50f5b-09ad-48b7-9df1-08657541b55e" containerName="extract"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030178    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7da49cce-9637-428c-a819-f878fb853e3e" containerName="extract"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030186    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="83168f79-3b2b-402e-8f90-6c712e45c39d" containerName="extract"
Jul 13 18:40:10.030849 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.030195    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfef29e3-76c2-4926-9db0-764c65cc63cd" containerName="extract"
Jul 13 18:40:10.033507 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.033483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.044462 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.044440    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86dfbf7594-92rxg"]
Jul 13 18:40:10.212328 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.212290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-console-serving-cert\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.212527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.212343    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-console-oauth-config\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.212527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.212360    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-service-ca\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.212527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.212377    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-console-config\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.212527 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.212473    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-trusted-ca-bundle\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.212760 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.212540    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-867zl\" (UniqueName: \"kubernetes.io/projected/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-kube-api-access-867zl\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.212760 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.212571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-oauth-serving-cert\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.313095 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.312996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-trusted-ca-bundle\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.313095 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.313066    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-867zl\" (UniqueName: \"kubernetes.io/projected/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-kube-api-access-867zl\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.313336 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.313109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-oauth-serving-cert\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.313336 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.313156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-console-serving-cert\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.313336 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.313208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-console-oauth-config\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.313336 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.313230    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-service-ca\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.313336 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.313256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-console-config\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.314014 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.313984    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-service-ca\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.314014 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.314009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-console-config\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.314168 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.314026    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-trusted-ca-bundle\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.314205 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.314164    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-oauth-serving-cert\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.315833 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.315808    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-console-serving-cert\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.315913 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.315846    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-console-oauth-config\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.323203 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.323183    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-867zl\" (UniqueName: \"kubernetes.io/projected/5a8681f7-ef88-44c5-9b99-bdaad3bf79b0-kube-api-access-867zl\") pod \"console-86dfbf7594-92rxg\" (UID: \"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0\") " pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.343154 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.343133    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:10.412236 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.412212    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z"
Jul 13 18:40:10.412585 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.412207    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z" event={"ID":"bfef29e3-76c2-4926-9db0-764c65cc63cd","Type":"ContainerDied","Data":"68a9eef8d86eae69644bf1f74c0e7404a88460ee51cfe32c12540e39df4f7cb7"}
Jul 13 18:40:10.412585 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.412336    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68a9eef8d86eae69644bf1f74c0e7404a88460ee51cfe32c12540e39df4f7cb7"
Jul 13 18:40:10.485574 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:10.485550    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86dfbf7594-92rxg"]
Jul 13 18:40:10.487480 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:40:10.487460    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8681f7_ef88_44c5_9b99_bdaad3bf79b0.slice/crio-2f37957a9c8024c3da70f49f24066bd2bb4370fccc3ad14d594113a3b562a26a WatchSource:0}: Error finding container 2f37957a9c8024c3da70f49f24066bd2bb4370fccc3ad14d594113a3b562a26a: Status 404 returned error can't find the container with id 2f37957a9c8024c3da70f49f24066bd2bb4370fccc3ad14d594113a3b562a26a
Jul 13 18:40:11.423205 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:11.423166    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86dfbf7594-92rxg" event={"ID":"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0","Type":"ContainerStarted","Data":"2c360967191590b07683fa9e22f9dbb6c250ce66a00ca361a56b64d882b7f30c"}
Jul 13 18:40:11.423599 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:11.423242    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86dfbf7594-92rxg" event={"ID":"5a8681f7-ef88-44c5-9b99-bdaad3bf79b0","Type":"ContainerStarted","Data":"2f37957a9c8024c3da70f49f24066bd2bb4370fccc3ad14d594113a3b562a26a"}
Jul 13 18:40:11.441183 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:11.441132    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86dfbf7594-92rxg" podStartSLOduration=1.441119661 podStartE2EDuration="1.441119661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:40:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:40:11.441066126 +0000 UTC m=+492.572354576" watchObservedRunningTime="2026-07-13 18:40:11.441119661 +0000 UTC m=+492.572408120"
Jul 13 18:40:20.343480 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:20.343444    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:20.343886 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:20.343529    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:20.347998 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:20.347976    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:20.468825 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:20.468797    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86dfbf7594-92rxg"
Jul 13 18:40:20.525272 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:20.525242    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-588c585d96-kt6mk"]
Jul 13 18:40:22.993841 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:22.993807    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw"]
Jul 13 18:40:22.997026 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:22.997011    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw"
Jul 13 18:40:23.008449 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:23.008432    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lgbxt\""
Jul 13 18:40:23.009293 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:23.009273    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 18:40:23.009293 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:23.009287    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 18:40:23.014386 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:23.014364    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7fvr\" (UniqueName: \"kubernetes.io/projected/56c20e66-c182-47d1-9615-f15ca870e130-kube-api-access-g7fvr\") pod \"authorino-operator-75cc8d5cb7-jwdcw\" (UID: \"56c20e66-c182-47d1-9615-f15ca870e130\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw"
Jul 13 18:40:23.019177 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:23.019155    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw"]
Jul 13 18:40:23.115217 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:23.115187    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7fvr\" (UniqueName: \"kubernetes.io/projected/56c20e66-c182-47d1-9615-f15ca870e130-kube-api-access-g7fvr\") pod \"authorino-operator-75cc8d5cb7-jwdcw\" (UID: \"56c20e66-c182-47d1-9615-f15ca870e130\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw"
Jul 13 18:40:23.143338 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:23.143300    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7fvr\" (UniqueName: \"kubernetes.io/projected/56c20e66-c182-47d1-9615-f15ca870e130-kube-api-access-g7fvr\") pod \"authorino-operator-75cc8d5cb7-jwdcw\" (UID: \"56c20e66-c182-47d1-9615-f15ca870e130\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw"
Jul 13 18:40:23.306491 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:23.306398    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw"
Jul 13 18:40:23.482543 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:23.482511    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw"]
Jul 13 18:40:23.484344 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:40:23.484319    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56c20e66_c182_47d1_9615_f15ca870e130.slice/crio-13fd6dfc63692f7ef2d114e3cd866496b496049e1a2c2c5a696b9c3d6d58cae4 WatchSource:0}: Error finding container 13fd6dfc63692f7ef2d114e3cd866496b496049e1a2c2c5a696b9c3d6d58cae4: Status 404 returned error can't find the container with id 13fd6dfc63692f7ef2d114e3cd866496b496049e1a2c2c5a696b9c3d6d58cae4
Jul 13 18:40:24.490723 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:24.490684    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw" event={"ID":"56c20e66-c182-47d1-9615-f15ca870e130","Type":"ContainerStarted","Data":"13fd6dfc63692f7ef2d114e3cd866496b496049e1a2c2c5a696b9c3d6d58cae4"}
Jul 13 18:40:26.503316 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:26.503283    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw" event={"ID":"56c20e66-c182-47d1-9615-f15ca870e130","Type":"ContainerStarted","Data":"e5290c4fc1e675bf857fb71e6637674a6d9f11cc862a77fc6ec592214ad22317"}
Jul 13 18:40:26.503691 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:26.503340    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw"
Jul 13 18:40:26.522523 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:26.522480    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw" podStartSLOduration=2.137066639 podStartE2EDuration="4.522465784s" totalImagesPullingTime="2.385399145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:40:26.521449513 +0000 UTC m=+507.652737979" watchObservedRunningTime="2026-07-13 18:40:26.522465784 +0000 UTC m=+507.653754244"
Jul 13 18:40:37.510885 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:37.510853    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jwdcw"
Jul 13 18:40:45.551395 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.551336    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-588c585d96-kt6mk" podUID="6ab2262e-8852-47fa-8c9d-bca8f43f480f" containerName="console" containerID="cri-o://a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936" gracePeriod=15
Jul 13 18:40:45.798561 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.798537    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:40:45.913863 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.913785    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-trusted-ca-bundle\") pod \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") "
Jul 13 18:40:45.913863 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.913850    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-serving-cert\") pod \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") "
Jul 13 18:40:45.914084 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.913885    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-service-ca\") pod \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") "
Jul 13 18:40:45.914084 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.913910    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-oauth-serving-cert\") pod \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") "
Jul 13 18:40:45.914084 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.913934    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9cn5\" (UniqueName: \"kubernetes.io/projected/6ab2262e-8852-47fa-8c9d-bca8f43f480f-kube-api-access-c9cn5\") pod \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") "
Jul 13 18:40:45.914084 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.913950    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-config\") pod \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") "
Jul 13 18:40:45.914084 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.914001    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-oauth-config\") pod \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\" (UID: \"6ab2262e-8852-47fa-8c9d-bca8f43f480f\") "
Jul 13 18:40:45.914420 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.914270    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6ab2262e-8852-47fa-8c9d-bca8f43f480f" (UID: "6ab2262e-8852-47fa-8c9d-bca8f43f480f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:40:45.914539 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.914427    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6ab2262e-8852-47fa-8c9d-bca8f43f480f" (UID: "6ab2262e-8852-47fa-8c9d-bca8f43f480f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:40:45.914539 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.914449    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-service-ca" (OuterVolumeSpecName: "service-ca") pod "6ab2262e-8852-47fa-8c9d-bca8f43f480f" (UID: "6ab2262e-8852-47fa-8c9d-bca8f43f480f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:40:45.914539 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.914475    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-config" (OuterVolumeSpecName: "console-config") pod "6ab2262e-8852-47fa-8c9d-bca8f43f480f" (UID: "6ab2262e-8852-47fa-8c9d-bca8f43f480f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 18:40:45.916252 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.916225    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6ab2262e-8852-47fa-8c9d-bca8f43f480f" (UID: "6ab2262e-8852-47fa-8c9d-bca8f43f480f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:40:45.916336 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.916267    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6ab2262e-8852-47fa-8c9d-bca8f43f480f" (UID: "6ab2262e-8852-47fa-8c9d-bca8f43f480f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 18:40:45.916336 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:45.916273    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ab2262e-8852-47fa-8c9d-bca8f43f480f-kube-api-access-c9cn5" (OuterVolumeSpecName: "kube-api-access-c9cn5") pod "6ab2262e-8852-47fa-8c9d-bca8f43f480f" (UID: "6ab2262e-8852-47fa-8c9d-bca8f43f480f"). InnerVolumeSpecName "kube-api-access-c9cn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 18:40:46.015052 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.015022    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-oauth-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:46.015052 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.015046    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-trusted-ca-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:46.015052 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.015055    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:46.015287 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.015064    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:46.015287 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.015073    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-oauth-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:46.015287 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.015081    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9cn5\" (UniqueName: \"kubernetes.io/projected/6ab2262e-8852-47fa-8c9d-bca8f43f480f-kube-api-access-c9cn5\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:46.015287 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.015090    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ab2262e-8852-47fa-8c9d-bca8f43f480f-console-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 18:40:46.604563 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.604526    2569 generic.go:356] "Generic (PLEG): container finished" podID="6ab2262e-8852-47fa-8c9d-bca8f43f480f" containerID="a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936" exitCode=0
Jul 13 18:40:46.604963 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.604573    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588c585d96-kt6mk"
Jul 13 18:40:46.604963 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.604578    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588c585d96-kt6mk" event={"ID":"6ab2262e-8852-47fa-8c9d-bca8f43f480f","Type":"ContainerDied","Data":"a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936"}
Jul 13 18:40:46.604963 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.604627    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588c585d96-kt6mk" event={"ID":"6ab2262e-8852-47fa-8c9d-bca8f43f480f","Type":"ContainerDied","Data":"15dcb30493257a87e9ff366bea50d5c77680e763558d00ecc4014427a4bcb20c"}
Jul 13 18:40:46.604963 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.604671    2569 scope.go:117] "RemoveContainer" containerID="a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936"
Jul 13 18:40:46.614313 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.614297    2569 scope.go:117] "RemoveContainer" containerID="a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936"
Jul 13 18:40:46.614579 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:40:46.614561    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936\": container with ID starting with a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936 not found: ID does not exist" containerID="a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936"
Jul 13 18:40:46.614645 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.614590    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936"} err="failed to get container status \"a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936\": rpc error: code = NotFound desc = could not find container \"a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936\": container with ID starting with a665031d5711970729386076bf031798ad12fb4511ec370cf5dcb2da2fc1c936 not found: ID does not exist"
Jul 13 18:40:46.628862 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.628832    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-588c585d96-kt6mk"]
Jul 13 18:40:46.633328 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:46.633307    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-588c585d96-kt6mk"]
Jul 13 18:40:47.355120 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:40:47.355078    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ab2262e-8852-47fa-8c9d-bca8f43f480f" path="/var/lib/kubelet/pods/6ab2262e-8852-47fa-8c9d-bca8f43f480f/volumes"
Jul 13 18:41:42.454789 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.454755    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:41:42.455288 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.455229    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ab2262e-8852-47fa-8c9d-bca8f43f480f" containerName="console"
Jul 13 18:41:42.455288 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.455248    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab2262e-8852-47fa-8c9d-bca8f43f480f" containerName="console"
Jul 13 18:41:42.455403 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.455327    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ab2262e-8852-47fa-8c9d-bca8f43f480f" containerName="console"
Jul 13 18:41:42.458479 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.458458    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"
Jul 13 18:41:42.461545 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.461525    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kkqv9\""
Jul 13 18:41:42.461819 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.461803    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 13 18:41:42.468408 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.468384    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:41:42.498554 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.498526    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:41:42.582852 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.582822    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98919ddd-2764-452c-958c-72fda9522a34-config-file\") pod \"limitador-limitador-54d6d988d5-ghhd9\" (UID: \"98919ddd-2764-452c-958c-72fda9522a34\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"
Jul 13 18:41:42.583006 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.582867    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qn8x\" (UniqueName: \"kubernetes.io/projected/98919ddd-2764-452c-958c-72fda9522a34-kube-api-access-7qn8x\") pod \"limitador-limitador-54d6d988d5-ghhd9\" (UID: \"98919ddd-2764-452c-958c-72fda9522a34\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"
Jul 13 18:41:42.683970 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.683936    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98919ddd-2764-452c-958c-72fda9522a34-config-file\") pod \"limitador-limitador-54d6d988d5-ghhd9\" (UID: \"98919ddd-2764-452c-958c-72fda9522a34\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"
Jul 13 18:41:42.683970 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.683976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qn8x\" (UniqueName: \"kubernetes.io/projected/98919ddd-2764-452c-958c-72fda9522a34-kube-api-access-7qn8x\") pod \"limitador-limitador-54d6d988d5-ghhd9\" (UID: \"98919ddd-2764-452c-958c-72fda9522a34\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"
Jul 13 18:41:42.684530 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.684511    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98919ddd-2764-452c-958c-72fda9522a34-config-file\") pod \"limitador-limitador-54d6d988d5-ghhd9\" (UID: \"98919ddd-2764-452c-958c-72fda9522a34\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"
Jul 13 18:41:42.693032 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.693003    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qn8x\" (UniqueName: \"kubernetes.io/projected/98919ddd-2764-452c-958c-72fda9522a34-kube-api-access-7qn8x\") pod \"limitador-limitador-54d6d988d5-ghhd9\" (UID: \"98919ddd-2764-452c-958c-72fda9522a34\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"
Jul 13 18:41:42.769298 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.769267    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"
Jul 13 18:41:42.900954 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:42.900915    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:41:42.903665 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:41:42.903623    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98919ddd_2764_452c_958c_72fda9522a34.slice/crio-736c4ae5563ad71ba71c2b4babdfcec027f43176b117ad8df3213a6c2148bb8b WatchSource:0}: Error finding container 736c4ae5563ad71ba71c2b4babdfcec027f43176b117ad8df3213a6c2148bb8b: Status 404 returned error can't find the container with id 736c4ae5563ad71ba71c2b4babdfcec027f43176b117ad8df3213a6c2148bb8b
Jul 13 18:41:43.889492 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:43.889456    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9" event={"ID":"98919ddd-2764-452c-958c-72fda9522a34","Type":"ContainerStarted","Data":"736c4ae5563ad71ba71c2b4babdfcec027f43176b117ad8df3213a6c2148bb8b"}
Jul 13 18:41:46.905834 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:46.905792    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9" event={"ID":"98919ddd-2764-452c-958c-72fda9522a34","Type":"ContainerStarted","Data":"eac2fabbc0c314d3702307a5fb6c95177f33d976e40023513f268e31cc783476"}
Jul 13 18:41:46.906235 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:46.905937    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"
Jul 13 18:41:46.923925 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:46.923879    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9" podStartSLOduration=1.7604514020000002 podStartE2EDuration="4.923865673s" totalImagesPullingTime="3.163414271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:41:46.921720449 +0000 UTC m=+588.053008905" watchObservedRunningTime="2026-07-13 18:41:46.923865673 +0000 UTC m=+588.055154132"
Jul 13 18:41:57.911724 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:57.911694    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"
Jul 13 18:41:59.258928 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:59.258902    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:41:59.259357 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:41:59.258966    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:43:01.563043 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:43:01.563007    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:43:17.832665 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:43:17.832611    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:43:26.639281 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:43:26.639246    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:43:31.442999 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:43:31.442965    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:43:45.030697 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:43:45.030646    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:44:10.078180 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.078109    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"]
Jul 13 18:44:10.081404 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.081382    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.084836 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.084814    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z76zx\""
Jul 13 18:44:10.084957 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.084846    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 13 18:44:10.084957 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.084846    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 13 18:44:10.084957 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.084848    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 13 18:44:10.090462 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.090440    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"]
Jul 13 18:44:10.133680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.133632    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.133680 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.133678    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.133906 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.133770    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43a08093-5986-4d29-adc7-ae2be5201c98-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.133906 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.133797    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.133906 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.133815    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.133906 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.133833    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkvbv\" (UniqueName: \"kubernetes.io/projected/43a08093-5986-4d29-adc7-ae2be5201c98-kube-api-access-hkvbv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.234737 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.234704    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.234737 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.234738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.234970 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.234799    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43a08093-5986-4d29-adc7-ae2be5201c98-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.234970 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.234817    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.234970 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.234880    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.234970 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.234931    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkvbv\" (UniqueName: \"kubernetes.io/projected/43a08093-5986-4d29-adc7-ae2be5201c98-kube-api-access-hkvbv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.235282 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.235256    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.235383 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.235322    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.235754 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.235731    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.237290 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.237271    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43a08093-5986-4d29-adc7-ae2be5201c98-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.237464 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.237436    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43a08093-5986-4d29-adc7-ae2be5201c98-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.243130 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.243109    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkvbv\" (UniqueName: \"kubernetes.io/projected/43a08093-5986-4d29-adc7-ae2be5201c98-kube-api-access-hkvbv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9n8b\" (UID: \"43a08093-5986-4d29-adc7-ae2be5201c98\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.392425 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.392345    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:10.527960 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.527933    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"]
Jul 13 18:44:10.529974 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:44:10.529946    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a08093_5986_4d29_adc7_ae2be5201c98.slice/crio-f3166fc4a4e83f570a9386849af2f26f433767a3f0904ed17f9c110909e5d950 WatchSource:0}: Error finding container f3166fc4a4e83f570a9386849af2f26f433767a3f0904ed17f9c110909e5d950: Status 404 returned error can't find the container with id f3166fc4a4e83f570a9386849af2f26f433767a3f0904ed17f9c110909e5d950
Jul 13 18:44:10.531290 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.531266    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 18:44:10.591292 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:10.591264    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b" event={"ID":"43a08093-5986-4d29-adc7-ae2be5201c98","Type":"ContainerStarted","Data":"f3166fc4a4e83f570a9386849af2f26f433767a3f0904ed17f9c110909e5d950"}
Jul 13 18:44:11.933116 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:11.933034    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:44:12.601884 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:12.601848    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b" event={"ID":"43a08093-5986-4d29-adc7-ae2be5201c98","Type":"ContainerStarted","Data":"995bb9680eb729553382d2daca003156c9a40ae081721dde51d2bada0d85c10e"}
Jul 13 18:44:12.621364 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:12.621322    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b" podStartSLOduration=1.539602651 podStartE2EDuration="2.621307593s" totalImagesPullingTime="1.081704942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:44:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:44:12.619504073 +0000 UTC m=+733.750792534" watchObservedRunningTime="2026-07-13 18:44:12.621307593 +0000 UTC m=+733.752596043"
Jul 13 18:44:20.393070 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:20.393039    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:20.393573 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:20.393097    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:20.405607 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:20.405584    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:20.651900 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:20.651812    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9n8b"
Jul 13 18:44:32.253458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:44:32.253422    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:46:11.444515 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:46:11.444474    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:46:21.032466 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:46:21.032429    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:46:31.542613 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:46:31.542584    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:46:40.883955 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:46:40.883921    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:46:51.632926 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:46:51.632846    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:46:59.289964 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:46:59.289935    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:46:59.298003 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:46:59.297973    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:47:59.741482 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:47:59.741440    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:48:08.840404 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:48:08.840368    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:48:17.341845 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:48:17.341810    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:48:30.543457 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:48:30.543363    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:48:41.241426 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:48:41.241391    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:48:51.539138 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:48:51.539104    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:49:02.937808 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:49:02.937776    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:49:10.634741 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:49:10.634708    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:49:18.431514 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:49:18.431482    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:49:26.242193 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:49:26.242156    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:49:38.550640 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:49:38.550610    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:49:49.933258 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:49:49.933159    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:50:13.438335 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:50:13.438299    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:50:23.643602 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:50:23.643562    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:50:39.539024 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:50:39.538988    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:50:49.291895 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:50:49.291860    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:50:58.532910 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:50:58.532872    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:51:07.836114 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:51:07.836076    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:51:17.043986 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:51:17.043949    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:51:59.326500 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:51:59.326470    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:51:59.329470 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:51:59.329450    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:53:08.643990 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:53:08.643953    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:53:20.542161 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:53:20.542129    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:53:28.049218 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:53:28.049185    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:53:42.259478 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:53:42.259446    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:53:52.542261 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:53:52.542225    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:54:01.744707 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:54:01.744672    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:54:12.244082 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:54:12.244037    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:54:22.057549 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:54:22.057459    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:54:33.044812 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:54:33.044780    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:54:42.364102 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:54:42.364055    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:54:55.878223 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:54:55.876751    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:55:20.063113 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:55:20.063079    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:55:27.640592 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:55:27.640554    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:55:37.638157 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:55:37.638121    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:55:47.247506 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:55:47.247466    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:55:59.254022 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:55:59.253947    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:56:09.854801 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:56:09.854767    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:56:20.628162 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:56:20.628129    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:56:29.735109 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:56:29.735072    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:56:38.967476 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:56:38.967443    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:56:50.043750 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:56:50.043715    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:56:59.359056 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:56:59.359029    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:56:59.366390 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:56:59.366369    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 18:56:59.741946 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:56:59.741915    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:57:12.732340 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:57:12.732310    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:57:21.350153 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:57:21.350084    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:57:31.042089 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:57:31.042052    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:57:42.744317 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:57:42.744276    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:57:54.036458 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:57:54.036427    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:58:04.147064 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:58:04.147024    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:58:16.338463 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:58:16.338432    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:58:30.036988 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:58:30.036952    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:58:41.942994 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:58:41.942952    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:58:52.534151 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:58:52.534056    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:59:03.532548 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:03.532514    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:59:12.934624 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:12.934586    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 18:59:14.943918 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:14.943886    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"]
Jul 13 18:59:14.947245 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:14.947231    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:14.950175 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:14.950154    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f7c6e726-serving-cert\""
Jul 13 18:59:14.950325 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:14.950280    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xfcfp\""
Jul 13 18:59:14.950947 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:14.950933    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 18:59:14.951016 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:14.950977    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 13 18:59:14.964841 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:14.964818    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"]
Jul 13 18:59:15.012665 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.012619    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-maas-api-tls\") pod \"maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.012825 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.012696    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgbts\" (UniqueName: \"kubernetes.io/projected/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-kube-api-access-wgbts\") pod \"maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.012825 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.012739    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-openshift-service-ca\") pod \"maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.113556 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.113514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-maas-api-tls\") pod \"maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.113773 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.113569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgbts\" (UniqueName: \"kubernetes.io/projected/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-kube-api-access-wgbts\") pod \"maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.113773 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.113609    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-openshift-service-ca\") pod \"maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.113773 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:59:15.113683    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-serving-cert: secret "maas-api-e2e-ait-f7c6e726-serving-cert" not found
Jul 13 18:59:15.113773 ip-10-0-139-206 kubenswrapper[2569]: E0713 18:59:15.113751    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-maas-api-tls podName:de43cfc3-c0cc-4a2c-b451-e7381937c1bb nodeName:}" failed. No retries permitted until 2026-07-13 18:59:15.613734396 +0000 UTC m=+1636.745022835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-maas-api-tls") pod "maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm" (UID: "de43cfc3-c0cc-4a2c-b451-e7381937c1bb") : secret "maas-api-e2e-ait-f7c6e726-serving-cert" not found
Jul 13 18:59:15.114332 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.114307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-openshift-service-ca\") pod \"maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.124102 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.124064    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgbts\" (UniqueName: \"kubernetes.io/projected/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-kube-api-access-wgbts\") pod \"maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.618790 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.618753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-maas-api-tls\") pod \"maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.621448 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.621414    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-maas-api-tls\") pod \"maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.857781 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.857724    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:15.993436 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.993410    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"]
Jul 13 18:59:15.995410 ip-10-0-139-206 kubenswrapper[2569]: W0713 18:59:15.995380    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde43cfc3_c0cc_4a2c_b451_e7381937c1bb.slice/crio-4c42fa9bac1a41b728a59e7d4e4065972324681b83e505ec80458bc2306195f6 WatchSource:0}: Error finding container 4c42fa9bac1a41b728a59e7d4e4065972324681b83e505ec80458bc2306195f6: Status 404 returned error can't find the container with id 4c42fa9bac1a41b728a59e7d4e4065972324681b83e505ec80458bc2306195f6
Jul 13 18:59:15.996603 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:15.996585    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 18:59:16.026146 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:16.026114    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm" event={"ID":"de43cfc3-c0cc-4a2c-b451-e7381937c1bb","Type":"ContainerStarted","Data":"4c42fa9bac1a41b728a59e7d4e4065972324681b83e505ec80458bc2306195f6"}
Jul 13 18:59:19.045148 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:19.045110    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm" event={"ID":"de43cfc3-c0cc-4a2c-b451-e7381937c1bb","Type":"ContainerStarted","Data":"ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def"}
Jul 13 18:59:19.045642 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:19.045234    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:19.062447 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:19.062401    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm" podStartSLOduration=2.549164317 podStartE2EDuration="5.062386107s" totalImagesPullingTime="2.51322179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:19.061373467 +0000 UTC m=+1640.192661929" watchObservedRunningTime="2026-07-13 18:59:19.062386107 +0000 UTC m=+1640.193674566"
Jul 13 18:59:25.056145 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:25.056114    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 18:59:25.436645 ip-10-0-139-206 kubenswrapper[2569]: I0713 18:59:25.436548    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:00:20.331025 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.330932    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"]
Jul 13 19:00:20.331578 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.331201    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm" podUID="de43cfc3-c0cc-4a2c-b451-e7381937c1bb" containerName="maas-api" containerID="cri-o://ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def" gracePeriod=30
Jul 13 19:00:20.591233 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.591169    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 19:00:20.684221 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.684184    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgbts\" (UniqueName: \"kubernetes.io/projected/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-kube-api-access-wgbts\") pod \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") "
Jul 13 19:00:20.684221 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.684226    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-maas-api-tls\") pod \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") "
Jul 13 19:00:20.684470 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.684267    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-openshift-service-ca\") pod \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\" (UID: \"de43cfc3-c0cc-4a2c-b451-e7381937c1bb\") "
Jul 13 19:00:20.684634 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.684611    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de43cfc3-c0cc-4a2c-b451-e7381937c1bb" (UID: "de43cfc3-c0cc-4a2c-b451-e7381937c1bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:00:20.686545 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.686528    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de43cfc3-c0cc-4a2c-b451-e7381937c1bb" (UID: "de43cfc3-c0cc-4a2c-b451-e7381937c1bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:00:20.686677 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.686636    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-kube-api-access-wgbts" (OuterVolumeSpecName: "kube-api-access-wgbts") pod "de43cfc3-c0cc-4a2c-b451-e7381937c1bb" (UID: "de43cfc3-c0cc-4a2c-b451-e7381937c1bb"). InnerVolumeSpecName "kube-api-access-wgbts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:00:20.785385 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.785352    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:00:20.785385 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.785380    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgbts\" (UniqueName: \"kubernetes.io/projected/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-kube-api-access-wgbts\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:00:20.785385 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:20.785391    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de43cfc3-c0cc-4a2c-b451-e7381937c1bb-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:00:21.353451 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:21.353411    2569 generic.go:356] "Generic (PLEG): container finished" podID="de43cfc3-c0cc-4a2c-b451-e7381937c1bb" containerID="ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def" exitCode=0
Jul 13 19:00:21.353451 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:21.353440    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"
Jul 13 19:00:21.353919 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:21.353486    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm" event={"ID":"de43cfc3-c0cc-4a2c-b451-e7381937c1bb","Type":"ContainerDied","Data":"ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def"}
Jul 13 19:00:21.353919 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:21.353533    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm" event={"ID":"de43cfc3-c0cc-4a2c-b451-e7381937c1bb","Type":"ContainerDied","Data":"4c42fa9bac1a41b728a59e7d4e4065972324681b83e505ec80458bc2306195f6"}
Jul 13 19:00:21.353919 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:21.353560    2569 scope.go:117] "RemoveContainer" containerID="ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def"
Jul 13 19:00:21.369254 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:21.369231    2569 scope.go:117] "RemoveContainer" containerID="ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def"
Jul 13 19:00:21.369513 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:00:21.369492    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def\": container with ID starting with ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def not found: ID does not exist" containerID="ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def"
Jul 13 19:00:21.369561 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:21.369521    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def"} err="failed to get container status \"ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def\": rpc error: code = NotFound desc = could not find container \"ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def\": container with ID starting with ef50d1d137ea5ef04ea8aa6822a39aae1a5ef7c174d334f9278bdebd9fdc3def not found: ID does not exist"
Jul 13 19:00:21.378439 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:21.378419    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"]
Jul 13 19:00:21.383817 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:21.383797    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7c6e726-7f86f79865-dnzbm"]
Jul 13 19:00:23.148275 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:23.148242    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:00:23.350925 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:23.350891    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de43cfc3-c0cc-4a2c-b451-e7381937c1bb" path="/var/lib/kubelet/pods/de43cfc3-c0cc-4a2c-b451-e7381937c1bb/volumes"
Jul 13 19:00:37.174369 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.174331    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"]
Jul 13 19:00:37.174877 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.174860    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de43cfc3-c0cc-4a2c-b451-e7381937c1bb" containerName="maas-api"
Jul 13 19:00:37.174928 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.174881    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="de43cfc3-c0cc-4a2c-b451-e7381937c1bb" containerName="maas-api"
Jul 13 19:00:37.174971 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.174958    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="de43cfc3-c0cc-4a2c-b451-e7381937c1bb" containerName="maas-api"
Jul 13 19:00:37.178202 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.178186    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:37.181320 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.181293    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-78327326-serving-cert\""
Jul 13 19:00:37.181447 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.181334    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 13 19:00:37.181933 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.181917    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xfcfp\""
Jul 13 19:00:37.182020 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.181916    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:00:37.187689 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.187645    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"]
Jul 13 19:00:37.329566 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.329535    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fjl9\" (UniqueName: \"kubernetes.io/projected/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-kube-api-access-5fjl9\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:37.329971 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.329571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:37.329971 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.329593    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-openshift-service-ca\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:37.430386 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.430307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fjl9\" (UniqueName: \"kubernetes.io/projected/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-kube-api-access-5fjl9\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:37.430386 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.430341    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:37.430386 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.430362    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-openshift-service-ca\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:37.430728 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:00:37.430487    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-serving-cert: secret "maas-api-e2e-migrate-78327326-serving-cert" not found
Jul 13 19:00:37.430728 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:00:37.430570    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls podName:9157e2e6-cdd6-4ea5-8f6a-216e389f6881 nodeName:}" failed. No retries permitted until 2026-07-13 19:00:37.930550599 +0000 UTC m=+1719.061839045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls") pod "maas-api-e2e-migrate-78327326-5676bc64db-xczs8" (UID: "9157e2e6-cdd6-4ea5-8f6a-216e389f6881") : secret "maas-api-e2e-migrate-78327326-serving-cert" not found
Jul 13 19:00:37.431046 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.431029    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-openshift-service-ca\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:37.439847 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.439819    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fjl9\" (UniqueName: \"kubernetes.io/projected/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-kube-api-access-5fjl9\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:37.934289 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:37.934253    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:37.934460 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:00:37.934405    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-serving-cert: secret "maas-api-e2e-migrate-78327326-serving-cert" not found
Jul 13 19:00:37.934507 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:00:37.934473    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls podName:9157e2e6-cdd6-4ea5-8f6a-216e389f6881 nodeName:}" failed. No retries permitted until 2026-07-13 19:00:38.93445546 +0000 UTC m=+1720.065743898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls") pod "maas-api-e2e-migrate-78327326-5676bc64db-xczs8" (UID: "9157e2e6-cdd6-4ea5-8f6a-216e389f6881") : secret "maas-api-e2e-migrate-78327326-serving-cert" not found
Jul 13 19:00:38.943471 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:38.943425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:38.945974 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:38.945954    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls\") pod \"maas-api-e2e-migrate-78327326-5676bc64db-xczs8\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:38.989884 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:38.989848    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:39.123895 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:39.123870    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"]
Jul 13 19:00:39.125325 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:00:39.125298    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9157e2e6_cdd6_4ea5_8f6a_216e389f6881.slice/crio-f065254ebed3020e9c80e5111b92e307e8fa1fdf61a580651a64c0a7cede53ce WatchSource:0}: Error finding container f065254ebed3020e9c80e5111b92e307e8fa1fdf61a580651a64c0a7cede53ce: Status 404 returned error can't find the container with id f065254ebed3020e9c80e5111b92e307e8fa1fdf61a580651a64c0a7cede53ce
Jul 13 19:00:39.442718 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:39.442684    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8" event={"ID":"9157e2e6-cdd6-4ea5-8f6a-216e389f6881","Type":"ContainerStarted","Data":"f065254ebed3020e9c80e5111b92e307e8fa1fdf61a580651a64c0a7cede53ce"}
Jul 13 19:00:40.449495 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:40.449460    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8" event={"ID":"9157e2e6-cdd6-4ea5-8f6a-216e389f6881","Type":"ContainerStarted","Data":"59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35"}
Jul 13 19:00:40.449901 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:40.449532    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:00:40.467701 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:40.467633    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8" podStartSLOduration=2.79771761 podStartE2EDuration="3.467616025s" totalImagesPullingTime="669.898415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:40.467218308 +0000 UTC m=+1721.598506772" watchObservedRunningTime="2026-07-13 19:00:40.467616025 +0000 UTC m=+1721.598904515"
Jul 13 19:00:42.327784 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:42.327745    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:00:46.460230 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:00:46.460203    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:01:44.642347 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:44.642307    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:01:47.692572 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:47.692538    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"]
Jul 13 19:01:47.692996 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:47.692744    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8" podUID="9157e2e6-cdd6-4ea5-8f6a-216e389f6881" containerName="maas-api" containerID="cri-o://59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35" gracePeriod=30
Jul 13 19:01:47.946387 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:47.946273    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:01:48.036866 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.036837    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fjl9\" (UniqueName: \"kubernetes.io/projected/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-kube-api-access-5fjl9\") pod \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") "
Jul 13 19:01:48.037058 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.036906    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-openshift-service-ca\") pod \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") "
Jul 13 19:01:48.037058 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.036951    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls\") pod \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\" (UID: \"9157e2e6-cdd6-4ea5-8f6a-216e389f6881\") "
Jul 13 19:01:48.037298 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.037276    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9157e2e6-cdd6-4ea5-8f6a-216e389f6881" (UID: "9157e2e6-cdd6-4ea5-8f6a-216e389f6881"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:01:48.039490 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.039461    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-kube-api-access-5fjl9" (OuterVolumeSpecName: "kube-api-access-5fjl9") pod "9157e2e6-cdd6-4ea5-8f6a-216e389f6881" (UID: "9157e2e6-cdd6-4ea5-8f6a-216e389f6881"). InnerVolumeSpecName "kube-api-access-5fjl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:01:48.039595 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.039491    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9157e2e6-cdd6-4ea5-8f6a-216e389f6881" (UID: "9157e2e6-cdd6-4ea5-8f6a-216e389f6881"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:01:48.138491 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.138455    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fjl9\" (UniqueName: \"kubernetes.io/projected/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-kube-api-access-5fjl9\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:01:48.138491 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.138484    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:01:48.138491 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.138494    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9157e2e6-cdd6-4ea5-8f6a-216e389f6881-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:01:48.785763 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.785728    2569 generic.go:356] "Generic (PLEG): container finished" podID="9157e2e6-cdd6-4ea5-8f6a-216e389f6881" containerID="59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35" exitCode=0
Jul 13 19:01:48.786256 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.785775    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"
Jul 13 19:01:48.786256 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.785811    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8" event={"ID":"9157e2e6-cdd6-4ea5-8f6a-216e389f6881","Type":"ContainerDied","Data":"59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35"}
Jul 13 19:01:48.786256 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.785864    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8" event={"ID":"9157e2e6-cdd6-4ea5-8f6a-216e389f6881","Type":"ContainerDied","Data":"f065254ebed3020e9c80e5111b92e307e8fa1fdf61a580651a64c0a7cede53ce"}
Jul 13 19:01:48.786256 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.785889    2569 scope.go:117] "RemoveContainer" containerID="59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35"
Jul 13 19:01:48.802041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.802001    2569 scope.go:117] "RemoveContainer" containerID="59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35"
Jul 13 19:01:48.802566 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:01:48.802539    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35\": container with ID starting with 59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35 not found: ID does not exist" containerID="59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35"
Jul 13 19:01:48.802644 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.802577    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35"} err="failed to get container status \"59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35\": rpc error: code = NotFound desc = could not find container \"59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35\": container with ID starting with 59bbf5f38a09ccad6714cf0c480d00b4e1a96a6330d1df95084d4c79349aed35 not found: ID does not exist"
Jul 13 19:01:48.821124 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.821101    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"]
Jul 13 19:01:48.825325 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:48.825304    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-78327326-5676bc64db-xczs8"]
Jul 13 19:01:49.350551 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:49.350517    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9157e2e6-cdd6-4ea5-8f6a-216e389f6881" path="/var/lib/kubelet/pods/9157e2e6-cdd6-4ea5-8f6a-216e389f6881/volumes"
Jul 13 19:01:56.643587 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.643555    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"]
Jul 13 19:01:56.643972 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.643959    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9157e2e6-cdd6-4ea5-8f6a-216e389f6881" containerName="maas-api"
Jul 13 19:01:56.644015 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.643973    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9157e2e6-cdd6-4ea5-8f6a-216e389f6881" containerName="maas-api"
Jul 13 19:01:56.644065 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.644056    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9157e2e6-cdd6-4ea5-8f6a-216e389f6881" containerName="maas-api"
Jul 13 19:01:56.648055 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.648036    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:56.650923 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.650905    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:01:56.651084 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.651067    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xfcfp\""
Jul 13 19:01:56.651864 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.651846    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-258c642c-serving-cert\""
Jul 13 19:01:56.651946 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.651874    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 13 19:01:56.658242 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.658218    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"]
Jul 13 19:01:56.708962 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.708933    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-openshift-service-ca\") pod \"maas-api-e2e-ait-258c642c-7d897fdf74-bljvw\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:56.709079 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.708980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx6sh\" (UniqueName: \"kubernetes.io/projected/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-kube-api-access-sx6sh\") pod \"maas-api-e2e-ait-258c642c-7d897fdf74-bljvw\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:56.709079 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.709027    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-maas-api-tls\") pod \"maas-api-e2e-ait-258c642c-7d897fdf74-bljvw\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:56.809980 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.809947    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-openshift-service-ca\") pod \"maas-api-e2e-ait-258c642c-7d897fdf74-bljvw\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:56.810135 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.810002    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx6sh\" (UniqueName: \"kubernetes.io/projected/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-kube-api-access-sx6sh\") pod \"maas-api-e2e-ait-258c642c-7d897fdf74-bljvw\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:56.810135 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.810036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-maas-api-tls\") pod \"maas-api-e2e-ait-258c642c-7d897fdf74-bljvw\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:56.810774 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.810747    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-openshift-service-ca\") pod \"maas-api-e2e-ait-258c642c-7d897fdf74-bljvw\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:56.812612 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.812590    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-maas-api-tls\") pod \"maas-api-e2e-ait-258c642c-7d897fdf74-bljvw\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:56.822341 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.822314    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx6sh\" (UniqueName: \"kubernetes.io/projected/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-kube-api-access-sx6sh\") pod \"maas-api-e2e-ait-258c642c-7d897fdf74-bljvw\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:56.958988 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:56.958905    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:57.115587 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:57.115559    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"]
Jul 13 19:01:57.118209 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:01:57.118179    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16dcb4fa_58c6_46d2_8e06_48f4d78a5dbc.slice/crio-bef598f9ce4796f46dd3e5b16f5354a916d12c76772d5ad8f3ca8121891e177a WatchSource:0}: Error finding container bef598f9ce4796f46dd3e5b16f5354a916d12c76772d5ad8f3ca8121891e177a: Status 404 returned error can't find the container with id bef598f9ce4796f46dd3e5b16f5354a916d12c76772d5ad8f3ca8121891e177a
Jul 13 19:01:57.833009 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:57.832973    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw" event={"ID":"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc","Type":"ContainerStarted","Data":"bef598f9ce4796f46dd3e5b16f5354a916d12c76772d5ad8f3ca8121891e177a"}
Jul 13 19:01:58.840156 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:58.840112    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw" event={"ID":"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc","Type":"ContainerStarted","Data":"3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9"}
Jul 13 19:01:58.840526 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:58.840259    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:01:58.862784 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:58.862735    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw" podStartSLOduration=1.87520949 podStartE2EDuration="2.862720411s" totalImagesPullingTime="987.510921ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:01:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:01:58.860771319 +0000 UTC m=+1799.992059779" watchObservedRunningTime="2026-07-13 19:01:58.862720411 +0000 UTC m=+1799.994008870"
Jul 13 19:01:59.396385 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:59.396355    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 19:01:59.401812 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:01:59.401793    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 19:02:04.851211 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:02:04.851177    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:02:05.138606 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:02:05.138524    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:03:02.181198 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.181160    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"]
Jul 13 19:03:02.181769 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.181373    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw" podUID="16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc" containerName="maas-api" containerID="cri-o://3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9" gracePeriod=30
Jul 13 19:03:02.431371 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.431311    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:03:02.569156 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.569129    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx6sh\" (UniqueName: \"kubernetes.io/projected/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-kube-api-access-sx6sh\") pod \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") "
Jul 13 19:03:02.569329 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.569164    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-maas-api-tls\") pod \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") "
Jul 13 19:03:02.569329 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.569191    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-openshift-service-ca\") pod \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\" (UID: \"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc\") "
Jul 13 19:03:02.569703 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.569669    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc" (UID: "16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:03:02.571304 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.571278    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-kube-api-access-sx6sh" (OuterVolumeSpecName: "kube-api-access-sx6sh") pod "16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc" (UID: "16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc"). InnerVolumeSpecName "kube-api-access-sx6sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:03:02.571389 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.571313    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc" (UID: "16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:03:02.669903 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.669863    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx6sh\" (UniqueName: \"kubernetes.io/projected/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-kube-api-access-sx6sh\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:03:02.669903 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.669901    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:03:02.669903 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:02.669911    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:03:03.164380 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:03.164349    2569 generic.go:356] "Generic (PLEG): container finished" podID="16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc" containerID="3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9" exitCode=0
Jul 13 19:03:03.164584 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:03.164403    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"
Jul 13 19:03:03.164584 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:03.164442    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw" event={"ID":"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc","Type":"ContainerDied","Data":"3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9"}
Jul 13 19:03:03.164584 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:03.164528    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw" event={"ID":"16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc","Type":"ContainerDied","Data":"bef598f9ce4796f46dd3e5b16f5354a916d12c76772d5ad8f3ca8121891e177a"}
Jul 13 19:03:03.164584 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:03.164552    2569 scope.go:117] "RemoveContainer" containerID="3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9"
Jul 13 19:03:03.173947 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:03.173918    2569 scope.go:117] "RemoveContainer" containerID="3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9"
Jul 13 19:03:03.174209 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:03:03.174189    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9\": container with ID starting with 3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9 not found: ID does not exist" containerID="3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9"
Jul 13 19:03:03.174277 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:03.174221    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9"} err="failed to get container status \"3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9\": rpc error: code = NotFound desc = could not find container \"3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9\": container with ID starting with 3966b97564b5d51892936acc989353f7ba90b9c5f09c7331875a0246a17fe6f9 not found: ID does not exist"
Jul 13 19:03:03.185552 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:03.185526    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"]
Jul 13 19:03:03.191155 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:03.191133    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-258c642c-7d897fdf74-bljvw"]
Jul 13 19:03:03.349939 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:03.349898    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc" path="/var/lib/kubelet/pods/16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc/volumes"
Jul 13 19:03:05.139690 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:05.139644    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:03:19.639067 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.638972    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"]
Jul 13 19:03:19.639554 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.639465    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc" containerName="maas-api"
Jul 13 19:03:19.639554 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.639483    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc" containerName="maas-api"
Jul 13 19:03:19.639713 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.639577    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="16dcb4fa-58c6-46d2-8e06-48f4d78a5dbc" containerName="maas-api"
Jul 13 19:03:19.642956 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.642933    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:19.646023 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.645998    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-695ca96e-serving-cert\""
Jul 13 19:03:19.646135 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.646081    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xfcfp\""
Jul 13 19:03:19.646538 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.646522    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:03:19.647328 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.647309    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 13 19:03:19.665379 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.665358    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"]
Jul 13 19:03:19.813034 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.813000    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z8mh\" (UniqueName: \"kubernetes.io/projected/e495593a-5295-4baa-b1e3-e72bd72a144a-kube-api-access-7z8mh\") pod \"maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:19.813277 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.813065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e495593a-5295-4baa-b1e3-e72bd72a144a-maas-api-tls\") pod \"maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:19.813277 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.813136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e495593a-5295-4baa-b1e3-e72bd72a144a-openshift-service-ca\") pod \"maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:19.913678 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.913570    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z8mh\" (UniqueName: \"kubernetes.io/projected/e495593a-5295-4baa-b1e3-e72bd72a144a-kube-api-access-7z8mh\") pod \"maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:19.913678 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.913628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e495593a-5295-4baa-b1e3-e72bd72a144a-maas-api-tls\") pod \"maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:19.913892 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.913681    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e495593a-5295-4baa-b1e3-e72bd72a144a-openshift-service-ca\") pod \"maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:19.913892 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:03:19.913769    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-serving-cert: secret "maas-api-e2e-derive-695ca96e-serving-cert" not found
Jul 13 19:03:19.913892 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:03:19.913830    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e495593a-5295-4baa-b1e3-e72bd72a144a-maas-api-tls podName:e495593a-5295-4baa-b1e3-e72bd72a144a nodeName:}" failed. No retries permitted until 2026-07-13 19:03:20.413814544 +0000 UTC m=+1881.545102983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e495593a-5295-4baa-b1e3-e72bd72a144a-maas-api-tls") pod "maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n" (UID: "e495593a-5295-4baa-b1e3-e72bd72a144a") : secret "maas-api-e2e-derive-695ca96e-serving-cert" not found
Jul 13 19:03:19.914301 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.914282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e495593a-5295-4baa-b1e3-e72bd72a144a-openshift-service-ca\") pod \"maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:19.928824 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:19.928795    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z8mh\" (UniqueName: \"kubernetes.io/projected/e495593a-5295-4baa-b1e3-e72bd72a144a-kube-api-access-7z8mh\") pod \"maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:20.419232 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:20.419197    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e495593a-5295-4baa-b1e3-e72bd72a144a-maas-api-tls\") pod \"maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:20.421832 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:20.421808    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e495593a-5295-4baa-b1e3-e72bd72a144a-maas-api-tls\") pod \"maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:20.553890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:20.553860    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:20.690118 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:20.689931    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"]
Jul 13 19:03:20.694903 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:03:20.694865    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode495593a_5295_4baa_b1e3_e72bd72a144a.slice/crio-ccd239e5764e2fca1ec76fd95c03621deb11db8cc6def92b9951fa211e3204a9 WatchSource:0}: Error finding container ccd239e5764e2fca1ec76fd95c03621deb11db8cc6def92b9951fa211e3204a9: Status 404 returned error can't find the container with id ccd239e5764e2fca1ec76fd95c03621deb11db8cc6def92b9951fa211e3204a9
Jul 13 19:03:21.259278 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:21.259243    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n" event={"ID":"e495593a-5295-4baa-b1e3-e72bd72a144a","Type":"ContainerStarted","Data":"c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836"}
Jul 13 19:03:21.259478 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:21.259298    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n" event={"ID":"e495593a-5295-4baa-b1e3-e72bd72a144a","Type":"ContainerStarted","Data":"ccd239e5764e2fca1ec76fd95c03621deb11db8cc6def92b9951fa211e3204a9"}
Jul 13 19:03:21.259478 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:21.259329    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:21.278131 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:21.278080    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n" podStartSLOduration=1.9078544229999999 podStartE2EDuration="2.278063463s" totalImagesPullingTime="370.20904ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:03:21.276901124 +0000 UTC m=+1882.408189612" watchObservedRunningTime="2026-07-13 19:03:21.278063463 +0000 UTC m=+1882.409351922"
Jul 13 19:03:27.252980 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:27.252944    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:03:27.271099 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:27.271074    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:34.330232 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.330197    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"]
Jul 13 19:03:34.330758 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.330443    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n" podUID="e495593a-5295-4baa-b1e3-e72bd72a144a" containerName="maas-api" containerID="cri-o://c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836" gracePeriod=30
Jul 13 19:03:34.585132 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.585070    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:34.626886 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.626852    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z8mh\" (UniqueName: \"kubernetes.io/projected/e495593a-5295-4baa-b1e3-e72bd72a144a-kube-api-access-7z8mh\") pod \"e495593a-5295-4baa-b1e3-e72bd72a144a\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") "
Jul 13 19:03:34.627070 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.626898    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e495593a-5295-4baa-b1e3-e72bd72a144a-maas-api-tls\") pod \"e495593a-5295-4baa-b1e3-e72bd72a144a\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") "
Jul 13 19:03:34.627070 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.626941    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e495593a-5295-4baa-b1e3-e72bd72a144a-openshift-service-ca\") pod \"e495593a-5295-4baa-b1e3-e72bd72a144a\" (UID: \"e495593a-5295-4baa-b1e3-e72bd72a144a\") "
Jul 13 19:03:34.627358 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.627333    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e495593a-5295-4baa-b1e3-e72bd72a144a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e495593a-5295-4baa-b1e3-e72bd72a144a" (UID: "e495593a-5295-4baa-b1e3-e72bd72a144a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:03:34.629777 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.629756    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e495593a-5295-4baa-b1e3-e72bd72a144a-kube-api-access-7z8mh" (OuterVolumeSpecName: "kube-api-access-7z8mh") pod "e495593a-5295-4baa-b1e3-e72bd72a144a" (UID: "e495593a-5295-4baa-b1e3-e72bd72a144a"). InnerVolumeSpecName "kube-api-access-7z8mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:03:34.629777 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.629755    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e495593a-5295-4baa-b1e3-e72bd72a144a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e495593a-5295-4baa-b1e3-e72bd72a144a" (UID: "e495593a-5295-4baa-b1e3-e72bd72a144a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:03:34.727579 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.727553    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z8mh\" (UniqueName: \"kubernetes.io/projected/e495593a-5295-4baa-b1e3-e72bd72a144a-kube-api-access-7z8mh\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:03:34.727579 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.727581    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e495593a-5295-4baa-b1e3-e72bd72a144a-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:03:34.727740 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:34.727591    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e495593a-5295-4baa-b1e3-e72bd72a144a-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:03:35.341244 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:35.341211    2569 generic.go:356] "Generic (PLEG): container finished" podID="e495593a-5295-4baa-b1e3-e72bd72a144a" containerID="c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836" exitCode=0
Jul 13 19:03:35.341709 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:35.341253    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"
Jul 13 19:03:35.341709 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:35.341290    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n" event={"ID":"e495593a-5295-4baa-b1e3-e72bd72a144a","Type":"ContainerDied","Data":"c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836"}
Jul 13 19:03:35.341709 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:35.341344    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n" event={"ID":"e495593a-5295-4baa-b1e3-e72bd72a144a","Type":"ContainerDied","Data":"ccd239e5764e2fca1ec76fd95c03621deb11db8cc6def92b9951fa211e3204a9"}
Jul 13 19:03:35.341709 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:35.341366    2569 scope.go:117] "RemoveContainer" containerID="c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836"
Jul 13 19:03:35.351722 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:35.351706    2569 scope.go:117] "RemoveContainer" containerID="c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836"
Jul 13 19:03:35.352005 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:03:35.351981    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836\": container with ID starting with c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836 not found: ID does not exist" containerID="c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836"
Jul 13 19:03:35.352087 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:35.352018    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836"} err="failed to get container status \"c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836\": rpc error: code = NotFound desc = could not find container \"c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836\": container with ID starting with c416c9b80a4d4193824a43b3e4065b9a5e3e9d0722d4273a757e875b8298f836 not found: ID does not exist"
Jul 13 19:03:35.366146 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:35.366122    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"]
Jul 13 19:03:35.368971 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:35.368948    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-695ca96e-79c754c7cd-6dw7n"]
Jul 13 19:03:37.355787 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:37.355753    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e495593a-5295-4baa-b1e3-e72bd72a144a" path="/var/lib/kubelet/pods/e495593a-5295-4baa-b1e3-e72bd72a144a/volumes"
Jul 13 19:03:38.135276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:03:38.135242    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:04:21.648280 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.648242    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"]
Jul 13 19:04:21.648728 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.648591    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e495593a-5295-4baa-b1e3-e72bd72a144a" containerName="maas-api"
Jul 13 19:04:21.648728 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.648602    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e495593a-5295-4baa-b1e3-e72bd72a144a" containerName="maas-api"
Jul 13 19:04:21.648728 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.648696    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e495593a-5295-4baa-b1e3-e72bd72a144a" containerName="maas-api"
Jul 13 19:04:21.651735 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.651719    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"
Jul 13 19:04:21.654433 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.654412    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:04:21.654572 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.654435    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:04:21.660350 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.660328    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"]
Jul 13 19:04:21.722164 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.722134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75f3c908-25de-443c-a8b1-f872a042cca2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-fzcl8\" (UID: \"75f3c908-25de-443c-a8b1-f872a042cca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"
Jul 13 19:04:21.823337 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.823302    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75f3c908-25de-443c-a8b1-f872a042cca2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-fzcl8\" (UID: \"75f3c908-25de-443c-a8b1-f872a042cca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"
Jul 13 19:04:21.823992 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.823972    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75f3c908-25de-443c-a8b1-f872a042cca2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-fzcl8\" (UID: \"75f3c908-25de-443c-a8b1-f872a042cca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"
Jul 13 19:04:21.963773 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:21.963682    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"
Jul 13 19:04:22.101621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:22.101595    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"]
Jul 13 19:04:22.103629 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:04:22.103603    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75f3c908_25de_443c_a8b1_f872a042cca2.slice/crio-34b918f318566800037d0a528a4d76128348b5af6002264e685a07f10c1bb336 WatchSource:0}: Error finding container 34b918f318566800037d0a528a4d76128348b5af6002264e685a07f10c1bb336: Status 404 returned error can't find the container with id 34b918f318566800037d0a528a4d76128348b5af6002264e685a07f10c1bb336
Jul 13 19:04:22.105404 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:22.105386    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:04:22.583285 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:22.583247    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8" event={"ID":"75f3c908-25de-443c-a8b1-f872a042cca2","Type":"ContainerStarted","Data":"34b918f318566800037d0a528a4d76128348b5af6002264e685a07f10c1bb336"}
Jul 13 19:04:32.633961 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:32.633924    2569 generic.go:356] "Generic (PLEG): container finished" podID="75f3c908-25de-443c-a8b1-f872a042cca2" containerID="4a5ded40b481295eb8a9ad9b01ce70d5bceb898a7079f86047a9d7475d9117ef" exitCode=6
Jul 13 19:04:32.634438 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:32.633979    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8" event={"ID":"75f3c908-25de-443c-a8b1-f872a042cca2","Type":"ContainerDied","Data":"4a5ded40b481295eb8a9ad9b01ce70d5bceb898a7079f86047a9d7475d9117ef"}
Jul 13 19:04:32.634438 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:32.634192    2569 scope.go:117] "RemoveContainer" containerID="4a5ded40b481295eb8a9ad9b01ce70d5bceb898a7079f86047a9d7475d9117ef"
Jul 13 19:04:33.642240 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:33.642207    2569 generic.go:356] "Generic (PLEG): container finished" podID="75f3c908-25de-443c-a8b1-f872a042cca2" containerID="c71dfe751b7ae7cf1fb12ac3699793ee0836300fd2295e1fb783e7c3bd77296f" exitCode=6
Jul 13 19:04:33.642719 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:33.642272    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8" event={"ID":"75f3c908-25de-443c-a8b1-f872a042cca2","Type":"ContainerDied","Data":"c71dfe751b7ae7cf1fb12ac3699793ee0836300fd2295e1fb783e7c3bd77296f"}
Jul 13 19:04:33.642719 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:33.642350    2569 scope.go:117] "RemoveContainer" containerID="4a5ded40b481295eb8a9ad9b01ce70d5bceb898a7079f86047a9d7475d9117ef"
Jul 13 19:04:33.642719 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:33.642520    2569 scope.go:117] "RemoveContainer" containerID="c71dfe751b7ae7cf1fb12ac3699793ee0836300fd2295e1fb783e7c3bd77296f"
Jul 13 19:04:33.642869 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:04:33.642754    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-fzcl8_odh-ai-gateway-infra(75f3c908-25de-443c-a8b1-f872a042cca2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8" podUID="75f3c908-25de-443c-a8b1-f872a042cca2"
Jul 13 19:04:34.649029 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:34.649006    2569 scope.go:117] "RemoveContainer" containerID="c71dfe751b7ae7cf1fb12ac3699793ee0836300fd2295e1fb783e7c3bd77296f"
Jul 13 19:04:34.649395 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:04:34.649183    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-fzcl8_odh-ai-gateway-infra(75f3c908-25de-443c-a8b1-f872a042cca2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8" podUID="75f3c908-25de-443c-a8b1-f872a042cca2"
Jul 13 19:04:49.346765 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:49.346737    2569 scope.go:117] "RemoveContainer" containerID="c71dfe751b7ae7cf1fb12ac3699793ee0836300fd2295e1fb783e7c3bd77296f"
Jul 13 19:04:49.724260 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:49.724151    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8" event={"ID":"75f3c908-25de-443c-a8b1-f872a042cca2","Type":"ContainerStarted","Data":"a1590a48781aaa0c06226fc131061a2136549be690b410f893d507110323833d"}
Jul 13 19:04:49.740710 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:49.740664    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8" podStartSLOduration=19.308588538 podStartE2EDuration="28.740633053s" totalImagesPullingTime="9.432044515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:04:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:04:49.738766223 +0000 UTC m=+1970.870054691" watchObservedRunningTime="2026-07-13 19:04:49.740633053 +0000 UTC m=+1970.871921512"
Jul 13 19:04:50.731224 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:50.731190    2569 generic.go:356] "Generic (PLEG): container finished" podID="75f3c908-25de-443c-a8b1-f872a042cca2" containerID="a1590a48781aaa0c06226fc131061a2136549be690b410f893d507110323833d" exitCode=6
Jul 13 19:04:50.731630 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:50.731262    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8" event={"ID":"75f3c908-25de-443c-a8b1-f872a042cca2","Type":"ContainerDied","Data":"a1590a48781aaa0c06226fc131061a2136549be690b410f893d507110323833d"}
Jul 13 19:04:50.731630 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:50.731321    2569 scope.go:117] "RemoveContainer" containerID="c71dfe751b7ae7cf1fb12ac3699793ee0836300fd2295e1fb783e7c3bd77296f"
Jul 13 19:04:50.731630 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:50.731600    2569 scope.go:117] "RemoveContainer" containerID="a1590a48781aaa0c06226fc131061a2136549be690b410f893d507110323833d"
Jul 13 19:04:50.731891 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:04:50.731865    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-fzcl8_odh-ai-gateway-infra(75f3c908-25de-443c-a8b1-f872a042cca2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8" podUID="75f3c908-25de-443c-a8b1-f872a042cca2"
Jul 13 19:04:50.755041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:50.755017    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"]
Jul 13 19:04:51.868063 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:51.868038    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"
Jul 13 19:04:51.989862 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:51.989746    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75f3c908-25de-443c-a8b1-f872a042cca2-maas-api-service-ca\") pod \"75f3c908-25de-443c-a8b1-f872a042cca2\" (UID: \"75f3c908-25de-443c-a8b1-f872a042cca2\") "
Jul 13 19:04:51.990129 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:51.990106    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75f3c908-25de-443c-a8b1-f872a042cca2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "75f3c908-25de-443c-a8b1-f872a042cca2" (UID: "75f3c908-25de-443c-a8b1-f872a042cca2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:04:52.091349 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:52.091305    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75f3c908-25de-443c-a8b1-f872a042cca2-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:04:52.743614 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:52.743580    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"
Jul 13 19:04:52.743791 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:52.743622    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8" event={"ID":"75f3c908-25de-443c-a8b1-f872a042cca2","Type":"ContainerDied","Data":"34b918f318566800037d0a528a4d76128348b5af6002264e685a07f10c1bb336"}
Jul 13 19:04:52.743791 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:52.743713    2569 scope.go:117] "RemoveContainer" containerID="a1590a48781aaa0c06226fc131061a2136549be690b410f893d507110323833d"
Jul 13 19:04:52.765378 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:52.765354    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"]
Jul 13 19:04:52.771367 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:52.771345    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fzcl8"]
Jul 13 19:04:53.352194 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:04:53.352162    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75f3c908-25de-443c-a8b1-f872a042cca2" path="/var/lib/kubelet/pods/75f3c908-25de-443c-a8b1-f872a042cca2/volumes"
Jul 13 19:05:13.836532 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:13.836494    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:05:22.949764 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:22.949725    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:05:30.643614 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.643574    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"]
Jul 13 19:05:30.644098 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.644068    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75f3c908-25de-443c-a8b1-f872a042cca2" containerName="revoke-keys"
Jul 13 19:05:30.644098 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.644095    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="75f3c908-25de-443c-a8b1-f872a042cca2" containerName="revoke-keys"
Jul 13 19:05:30.644220 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.644107    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75f3c908-25de-443c-a8b1-f872a042cca2" containerName="revoke-keys"
Jul 13 19:05:30.644220 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.644116    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="75f3c908-25de-443c-a8b1-f872a042cca2" containerName="revoke-keys"
Jul 13 19:05:30.644315 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.644229    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="75f3c908-25de-443c-a8b1-f872a042cca2" containerName="revoke-keys"
Jul 13 19:05:30.644315 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.644243    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="75f3c908-25de-443c-a8b1-f872a042cca2" containerName="revoke-keys"
Jul 13 19:05:30.644315 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.644253    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="75f3c908-25de-443c-a8b1-f872a042cca2" containerName="revoke-keys"
Jul 13 19:05:30.651131 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.651111    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:30.656130 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.656107    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xfcfp\""
Jul 13 19:05:30.656268 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.656247    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:05:30.656877 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.656856    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7577a6d1-serving-cert\""
Jul 13 19:05:30.656877 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.656874    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 13 19:05:30.668122 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.668093    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"]
Jul 13 19:05:30.730544 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.730515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d919d925-0abc-4071-b905-38a8d33a5fb6-openshift-service-ca\") pod \"maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:30.730738 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.730574    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksbhx\" (UniqueName: \"kubernetes.io/projected/d919d925-0abc-4071-b905-38a8d33a5fb6-kube-api-access-ksbhx\") pod \"maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:30.730738 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.730605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d919d925-0abc-4071-b905-38a8d33a5fb6-maas-api-tls\") pod \"maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:30.831832 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.831800    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d919d925-0abc-4071-b905-38a8d33a5fb6-openshift-service-ca\") pod \"maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:30.832023 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.831849    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksbhx\" (UniqueName: \"kubernetes.io/projected/d919d925-0abc-4071-b905-38a8d33a5fb6-kube-api-access-ksbhx\") pod \"maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:30.832023 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.831886    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d919d925-0abc-4071-b905-38a8d33a5fb6-maas-api-tls\") pod \"maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:30.832023 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:05:30.831985    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-serving-cert: secret "maas-api-e2e-mt-7577a6d1-serving-cert" not found
Jul 13 19:05:30.832210 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:05:30.832048    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d919d925-0abc-4071-b905-38a8d33a5fb6-maas-api-tls podName:d919d925-0abc-4071-b905-38a8d33a5fb6 nodeName:}" failed. No retries permitted until 2026-07-13 19:05:31.332032254 +0000 UTC m=+2012.463320692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d919d925-0abc-4071-b905-38a8d33a5fb6-maas-api-tls") pod "maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2" (UID: "d919d925-0abc-4071-b905-38a8d33a5fb6") : secret "maas-api-e2e-mt-7577a6d1-serving-cert" not found
Jul 13 19:05:30.832498 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.832476    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d919d925-0abc-4071-b905-38a8d33a5fb6-openshift-service-ca\") pod \"maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:30.850626 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:30.850602    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksbhx\" (UniqueName: \"kubernetes.io/projected/d919d925-0abc-4071-b905-38a8d33a5fb6-kube-api-access-ksbhx\") pod \"maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:31.336837 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.336796    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d919d925-0abc-4071-b905-38a8d33a5fb6-maas-api-tls\") pod \"maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:31.339444 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.339419    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d919d925-0abc-4071-b905-38a8d33a5fb6-maas-api-tls\") pod \"maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:31.561982 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.561945    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:31.703815 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.703787    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"]
Jul 13 19:05:31.705887 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.705865    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"]
Jul 13 19:05:31.706195 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:05:31.706174    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd919d925_0abc_4071_b905_38a8d33a5fb6.slice/crio-d4cf222f11779544fd2b5ef158c6ff3ccc3af49c7948b5a0b7054f5aff2ee051 WatchSource:0}: Error finding container d4cf222f11779544fd2b5ef158c6ff3ccc3af49c7948b5a0b7054f5aff2ee051: Status 404 returned error can't find the container with id d4cf222f11779544fd2b5ef158c6ff3ccc3af49c7948b5a0b7054f5aff2ee051
Jul 13 19:05:31.706283 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.706268    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75f3c908-25de-443c-a8b1-f872a042cca2" containerName="revoke-keys"
Jul 13 19:05:31.706321 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.706286    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="75f3c908-25de-443c-a8b1-f872a042cca2" containerName="revoke-keys"
Jul 13 19:05:31.710494 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.710417    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"
Jul 13 19:05:31.712863 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.712839    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:05:31.717530 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.717510    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"]
Jul 13 19:05:31.740980 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.740956    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e833c54c-7b7b-4514-b5db-185c3b320d56-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-lbbgq\" (UID: \"e833c54c-7b7b-4514-b5db-185c3b320d56\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"
Jul 13 19:05:31.841478 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.841444    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e833c54c-7b7b-4514-b5db-185c3b320d56-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-lbbgq\" (UID: \"e833c54c-7b7b-4514-b5db-185c3b320d56\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"
Jul 13 19:05:31.842066 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.842046    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e833c54c-7b7b-4514-b5db-185c3b320d56-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-lbbgq\" (UID: \"e833c54c-7b7b-4514-b5db-185c3b320d56\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"
Jul 13 19:05:31.938713 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:31.938610    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2" event={"ID":"d919d925-0abc-4071-b905-38a8d33a5fb6","Type":"ContainerStarted","Data":"d4cf222f11779544fd2b5ef158c6ff3ccc3af49c7948b5a0b7054f5aff2ee051"}
Jul 13 19:05:32.023758 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:32.023729    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"
Jul 13 19:05:32.363807 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:32.363783    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"]
Jul 13 19:05:32.365525 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:05:32.365499    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode833c54c_7b7b_4514_b5db_185c3b320d56.slice/crio-c7042d0c64b327e8580aae51804a2dcd5e4a0fa8a3533597e1dd863ea22767c3 WatchSource:0}: Error finding container c7042d0c64b327e8580aae51804a2dcd5e4a0fa8a3533597e1dd863ea22767c3: Status 404 returned error can't find the container with id c7042d0c64b327e8580aae51804a2dcd5e4a0fa8a3533597e1dd863ea22767c3
Jul 13 19:05:32.945051 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:32.945012    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq" event={"ID":"e833c54c-7b7b-4514-b5db-185c3b320d56","Type":"ContainerStarted","Data":"b014000785f2cb586fa4625a63bd6ae31808d826a4140dff091bf11e57ca254c"}
Jul 13 19:05:32.945498 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:32.945065    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq" event={"ID":"e833c54c-7b7b-4514-b5db-185c3b320d56","Type":"ContainerStarted","Data":"c7042d0c64b327e8580aae51804a2dcd5e4a0fa8a3533597e1dd863ea22767c3"}
Jul 13 19:05:32.946703 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:32.946673    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2" event={"ID":"d919d925-0abc-4071-b905-38a8d33a5fb6","Type":"ContainerStarted","Data":"d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c"}
Jul 13 19:05:32.946820 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:32.946731    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:32.960331 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:32.960284    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq" podStartSLOduration=1.960268735 podStartE2EDuration="1.960268735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:05:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:05:32.959837235 +0000 UTC m=+2014.091125697" watchObservedRunningTime="2026-07-13 19:05:32.960268735 +0000 UTC m=+2014.091557194"
Jul 13 19:05:32.980834 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:32.980791    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2" podStartSLOduration=2.243518207 podStartE2EDuration="2.98077574s" totalImagesPullingTime="737.257533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:05:32.979639797 +0000 UTC m=+2014.110928256" watchObservedRunningTime="2026-07-13 19:05:32.98077574 +0000 UTC m=+2014.112064199"
Jul 13 19:05:33.953990 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:33.953957    2569 generic.go:356] "Generic (PLEG): container finished" podID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerID="b014000785f2cb586fa4625a63bd6ae31808d826a4140dff091bf11e57ca254c" exitCode=6
Jul 13 19:05:33.954404 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:33.954053    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq" event={"ID":"e833c54c-7b7b-4514-b5db-185c3b320d56","Type":"ContainerDied","Data":"b014000785f2cb586fa4625a63bd6ae31808d826a4140dff091bf11e57ca254c"}
Jul 13 19:05:33.954404 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:33.954176    2569 scope.go:117] "RemoveContainer" containerID="b014000785f2cb586fa4625a63bd6ae31808d826a4140dff091bf11e57ca254c"
Jul 13 19:05:34.960872 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:34.960773    2569 generic.go:356] "Generic (PLEG): container finished" podID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerID="ebe83855dbcf7b1dd77b403fecfb0ba07f1f84d8ef990198c2bd9fd8894eb958" exitCode=6
Jul 13 19:05:34.960872 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:34.960855    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq" event={"ID":"e833c54c-7b7b-4514-b5db-185c3b320d56","Type":"ContainerDied","Data":"ebe83855dbcf7b1dd77b403fecfb0ba07f1f84d8ef990198c2bd9fd8894eb958"}
Jul 13 19:05:34.961333 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:34.960917    2569 scope.go:117] "RemoveContainer" containerID="b014000785f2cb586fa4625a63bd6ae31808d826a4140dff091bf11e57ca254c"
Jul 13 19:05:34.961333 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:34.961153    2569 scope.go:117] "RemoveContainer" containerID="ebe83855dbcf7b1dd77b403fecfb0ba07f1f84d8ef990198c2bd9fd8894eb958"
Jul 13 19:05:34.961402 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:05:34.961385    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-lbbgq_odh-ai-gateway-infra(e833c54c-7b7b-4514-b5db-185c3b320d56)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56"
Jul 13 19:05:35.059523 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:35.059481    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:05:35.967871 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:35.967843    2569 scope.go:117] "RemoveContainer" containerID="ebe83855dbcf7b1dd77b403fecfb0ba07f1f84d8ef990198c2bd9fd8894eb958"
Jul 13 19:05:35.968256 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:05:35.968034    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-lbbgq_odh-ai-gateway-infra(e833c54c-7b7b-4514-b5db-185c3b320d56)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56"
Jul 13 19:05:38.960288 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:38.960259    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:42.929325 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:42.929289    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq"]
Jul 13 19:05:42.933983 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:42.933967    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq"
Jul 13 19:05:42.939672 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:42.939624    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq"]
Jul 13 19:05:43.034930 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:43.034888    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35d0c37-6215-4bfb-9344-fdca984d2e55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq\" (UID: \"f35d0c37-6215-4bfb-9344-fdca984d2e55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq"
Jul 13 19:05:43.135792 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:43.135753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35d0c37-6215-4bfb-9344-fdca984d2e55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq\" (UID: \"f35d0c37-6215-4bfb-9344-fdca984d2e55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq"
Jul 13 19:05:43.136362 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:43.136342    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35d0c37-6215-4bfb-9344-fdca984d2e55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq\" (UID: \"f35d0c37-6215-4bfb-9344-fdca984d2e55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq"
Jul 13 19:05:43.245978 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:43.245939    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq"
Jul 13 19:05:43.379834 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:43.379810    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq"]
Jul 13 19:05:43.381508 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:05:43.381479    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35d0c37_6215_4bfb_9344_fdca984d2e55.slice/crio-d1dfbb4aa8e2e0c35086b560eb5faf04a9def7effddcbe8ff1d457b563c25e4e WatchSource:0}: Error finding container d1dfbb4aa8e2e0c35086b560eb5faf04a9def7effddcbe8ff1d457b563c25e4e: Status 404 returned error can't find the container with id d1dfbb4aa8e2e0c35086b560eb5faf04a9def7effddcbe8ff1d457b563c25e4e
Jul 13 19:05:44.009645 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:44.009610    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq" event={"ID":"f35d0c37-6215-4bfb-9344-fdca984d2e55","Type":"ContainerStarted","Data":"30da3b6a810f0fef49eebad15a3b894d1e61ae632e302de6e4b4c440d7197d54"}
Jul 13 19:05:44.010018 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:44.009684    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq" event={"ID":"f35d0c37-6215-4bfb-9344-fdca984d2e55","Type":"ContainerStarted","Data":"d1dfbb4aa8e2e0c35086b560eb5faf04a9def7effddcbe8ff1d457b563c25e4e"}
Jul 13 19:05:44.025989 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:44.025941    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq" podStartSLOduration=2.025926311 podStartE2EDuration="2.025926311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:05:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:05:44.024510432 +0000 UTC m=+2025.155798891" watchObservedRunningTime="2026-07-13 19:05:44.025926311 +0000 UTC m=+2025.157214815"
Jul 13 19:05:45.016516 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:45.016482    2569 generic.go:356] "Generic (PLEG): container finished" podID="f35d0c37-6215-4bfb-9344-fdca984d2e55" containerID="30da3b6a810f0fef49eebad15a3b894d1e61ae632e302de6e4b4c440d7197d54" exitCode=0
Jul 13 19:05:45.016994 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:45.016574    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq" event={"ID":"f35d0c37-6215-4bfb-9344-fdca984d2e55","Type":"ContainerDied","Data":"30da3b6a810f0fef49eebad15a3b894d1e61ae632e302de6e4b4c440d7197d54"}
Jul 13 19:05:46.143686 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:46.143644    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq"
Jul 13 19:05:46.264190 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:46.264142    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35d0c37-6215-4bfb-9344-fdca984d2e55-maas-api-service-ca\") pod \"f35d0c37-6215-4bfb-9344-fdca984d2e55\" (UID: \"f35d0c37-6215-4bfb-9344-fdca984d2e55\") "
Jul 13 19:05:46.264487 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:46.264466    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35d0c37-6215-4bfb-9344-fdca984d2e55-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f35d0c37-6215-4bfb-9344-fdca984d2e55" (UID: "f35d0c37-6215-4bfb-9344-fdca984d2e55"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:05:46.365581 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:46.365483    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35d0c37-6215-4bfb-9344-fdca984d2e55-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:05:47.027521 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:47.027489    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq"
Jul 13 19:05:47.027711 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:47.027521    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq" event={"ID":"f35d0c37-6215-4bfb-9344-fdca984d2e55","Type":"ContainerDied","Data":"d1dfbb4aa8e2e0c35086b560eb5faf04a9def7effddcbe8ff1d457b563c25e4e"}
Jul 13 19:05:47.027711 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:47.027591    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1dfbb4aa8e2e0c35086b560eb5faf04a9def7effddcbe8ff1d457b563c25e4e"
Jul 13 19:05:50.344475 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:50.344448    2569 scope.go:117] "RemoveContainer" containerID="ebe83855dbcf7b1dd77b403fecfb0ba07f1f84d8ef990198c2bd9fd8894eb958"
Jul 13 19:05:51.047770 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:51.047734    2569 generic.go:356] "Generic (PLEG): container finished" podID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerID="555e501d58a5e9d2800a0b98f44b084a7ffdf339cef282af5c6bba3fc2ab2cec" exitCode=6
Jul 13 19:05:51.047953 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:51.047787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq" event={"ID":"e833c54c-7b7b-4514-b5db-185c3b320d56","Type":"ContainerDied","Data":"555e501d58a5e9d2800a0b98f44b084a7ffdf339cef282af5c6bba3fc2ab2cec"}
Jul 13 19:05:51.047953 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:51.047848    2569 scope.go:117] "RemoveContainer" containerID="ebe83855dbcf7b1dd77b403fecfb0ba07f1f84d8ef990198c2bd9fd8894eb958"
Jul 13 19:05:51.048173 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:51.048157    2569 scope.go:117] "RemoveContainer" containerID="555e501d58a5e9d2800a0b98f44b084a7ffdf339cef282af5c6bba3fc2ab2cec"
Jul 13 19:05:51.048396 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:05:51.048352    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-lbbgq_odh-ai-gateway-infra(e833c54c-7b7b-4514-b5db-185c3b320d56)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56"
Jul 13 19:05:52.073778 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:52.073745    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"]
Jul 13 19:05:52.205681 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:52.205638    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"
Jul 13 19:05:52.314931 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:52.314899    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e833c54c-7b7b-4514-b5db-185c3b320d56-maas-api-service-ca\") pod \"e833c54c-7b7b-4514-b5db-185c3b320d56\" (UID: \"e833c54c-7b7b-4514-b5db-185c3b320d56\") "
Jul 13 19:05:52.315252 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:52.315230    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e833c54c-7b7b-4514-b5db-185c3b320d56-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e833c54c-7b7b-4514-b5db-185c3b320d56" (UID: "e833c54c-7b7b-4514-b5db-185c3b320d56"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:05:52.415802 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:52.415707    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e833c54c-7b7b-4514-b5db-185c3b320d56-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:05:53.060031 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.060004    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"
Jul 13 19:05:53.060256 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.060038    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq" event={"ID":"e833c54c-7b7b-4514-b5db-185c3b320d56","Type":"ContainerDied","Data":"c7042d0c64b327e8580aae51804a2dcd5e4a0fa8a3533597e1dd863ea22767c3"}
Jul 13 19:05:53.060256 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.060104    2569 scope.go:117] "RemoveContainer" containerID="555e501d58a5e9d2800a0b98f44b084a7ffdf339cef282af5c6bba3fc2ab2cec"
Jul 13 19:05:53.081065 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.081039    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"]
Jul 13 19:05:53.091175 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.089318    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lbbgq"]
Jul 13 19:05:53.350573 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.350494    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56" path="/var/lib/kubelet/pods/e833c54c-7b7b-4514-b5db-185c3b320d56/volumes"
Jul 13 19:05:53.379100 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.379028    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"]
Jul 13 19:05:53.379328 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.379300    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2" podUID="d919d925-0abc-4071-b905-38a8d33a5fb6" containerName="maas-api" containerID="cri-o://d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c" gracePeriod=30
Jul 13 19:05:53.626441 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.626415    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:53.725992 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.725957    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksbhx\" (UniqueName: \"kubernetes.io/projected/d919d925-0abc-4071-b905-38a8d33a5fb6-kube-api-access-ksbhx\") pod \"d919d925-0abc-4071-b905-38a8d33a5fb6\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") "
Jul 13 19:05:53.726182 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.726052    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d919d925-0abc-4071-b905-38a8d33a5fb6-maas-api-tls\") pod \"d919d925-0abc-4071-b905-38a8d33a5fb6\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") "
Jul 13 19:05:53.726182 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.726116    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d919d925-0abc-4071-b905-38a8d33a5fb6-openshift-service-ca\") pod \"d919d925-0abc-4071-b905-38a8d33a5fb6\" (UID: \"d919d925-0abc-4071-b905-38a8d33a5fb6\") "
Jul 13 19:05:53.726473 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.726446    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d919d925-0abc-4071-b905-38a8d33a5fb6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d919d925-0abc-4071-b905-38a8d33a5fb6" (UID: "d919d925-0abc-4071-b905-38a8d33a5fb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:05:53.728273 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.728245    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d919d925-0abc-4071-b905-38a8d33a5fb6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d919d925-0abc-4071-b905-38a8d33a5fb6" (UID: "d919d925-0abc-4071-b905-38a8d33a5fb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:05:53.728376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.728342    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d919d925-0abc-4071-b905-38a8d33a5fb6-kube-api-access-ksbhx" (OuterVolumeSpecName: "kube-api-access-ksbhx") pod "d919d925-0abc-4071-b905-38a8d33a5fb6" (UID: "d919d925-0abc-4071-b905-38a8d33a5fb6"). InnerVolumeSpecName "kube-api-access-ksbhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:05:53.826986 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.826952    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d919d925-0abc-4071-b905-38a8d33a5fb6-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:05:53.826986 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.826980    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksbhx\" (UniqueName: \"kubernetes.io/projected/d919d925-0abc-4071-b905-38a8d33a5fb6-kube-api-access-ksbhx\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:05:53.826986 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:53.826991    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d919d925-0abc-4071-b905-38a8d33a5fb6-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:05:54.067406 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:54.067373    2569 generic.go:356] "Generic (PLEG): container finished" podID="d919d925-0abc-4071-b905-38a8d33a5fb6" containerID="d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c" exitCode=0
Jul 13 19:05:54.067566 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:54.067420    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"
Jul 13 19:05:54.067566 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:54.067461    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2" event={"ID":"d919d925-0abc-4071-b905-38a8d33a5fb6","Type":"ContainerDied","Data":"d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c"}
Jul 13 19:05:54.067566 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:54.067516    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2" event={"ID":"d919d925-0abc-4071-b905-38a8d33a5fb6","Type":"ContainerDied","Data":"d4cf222f11779544fd2b5ef158c6ff3ccc3af49c7948b5a0b7054f5aff2ee051"}
Jul 13 19:05:54.067566 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:54.067546    2569 scope.go:117] "RemoveContainer" containerID="d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c"
Jul 13 19:05:54.076705 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:54.076690    2569 scope.go:117] "RemoveContainer" containerID="d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c"
Jul 13 19:05:54.076956 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:05:54.076937    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c\": container with ID starting with d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c not found: ID does not exist" containerID="d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c"
Jul 13 19:05:54.077019 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:54.076968    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c"} err="failed to get container status \"d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c\": rpc error: code = NotFound desc = could not find container \"d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c\": container with ID starting with d5626710df8e55054fe53241a457026af4272d42ffc0fe3ccbe290b55d8d401c not found: ID does not exist"
Jul 13 19:05:54.089174 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:54.089143    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"]
Jul 13 19:05:54.093198 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:54.093178    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7577a6d1-7c8f978fcb-b26f2"]
Jul 13 19:05:55.350434 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:55.350390    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d919d925-0abc-4071-b905-38a8d33a5fb6" path="/var/lib/kubelet/pods/d919d925-0abc-4071-b905-38a8d33a5fb6/volumes"
Jul 13 19:05:56.236717 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:05:56.236686    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:06:31.768038 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768003    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"]
Jul 13 19:06:31.768463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768344    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35d0c37-6215-4bfb-9344-fdca984d2e55" containerName="revoke-keys"
Jul 13 19:06:31.768463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768356    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35d0c37-6215-4bfb-9344-fdca984d2e55" containerName="revoke-keys"
Jul 13 19:06:31.768463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768366    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerName="revoke-keys"
Jul 13 19:06:31.768463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768373    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerName="revoke-keys"
Jul 13 19:06:31.768463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768382    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerName="revoke-keys"
Jul 13 19:06:31.768463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768388    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerName="revoke-keys"
Jul 13 19:06:31.768463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768401    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerName="revoke-keys"
Jul 13 19:06:31.768463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768405    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerName="revoke-keys"
Jul 13 19:06:31.768463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768413    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d919d925-0abc-4071-b905-38a8d33a5fb6" containerName="maas-api"
Jul 13 19:06:31.768463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768419    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d919d925-0abc-4071-b905-38a8d33a5fb6" containerName="maas-api"
Jul 13 19:06:31.768789 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768470    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerName="revoke-keys"
Jul 13 19:06:31.768789 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768479    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerName="revoke-keys"
Jul 13 19:06:31.768789 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768485    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f35d0c37-6215-4bfb-9344-fdca984d2e55" containerName="revoke-keys"
Jul 13 19:06:31.768789 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768491    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e833c54c-7b7b-4514-b5db-185c3b320d56" containerName="revoke-keys"
Jul 13 19:06:31.768789 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.768496    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d919d925-0abc-4071-b905-38a8d33a5fb6" containerName="maas-api"
Jul 13 19:06:31.771352 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.771336    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"
Jul 13 19:06:31.774164 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.774147    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:06:31.774284 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.774183    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:06:31.780003 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.779980    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"]
Jul 13 19:06:31.839471 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.839436    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff39b2ab-d7d8-4e50-b454-94f15dffa6a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-mq5vp\" (UID: \"ff39b2ab-d7d8-4e50-b454-94f15dffa6a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"
Jul 13 19:06:31.940251 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.940210    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff39b2ab-d7d8-4e50-b454-94f15dffa6a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-mq5vp\" (UID: \"ff39b2ab-d7d8-4e50-b454-94f15dffa6a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"
Jul 13 19:06:31.940874 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:31.940852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff39b2ab-d7d8-4e50-b454-94f15dffa6a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-mq5vp\" (UID: \"ff39b2ab-d7d8-4e50-b454-94f15dffa6a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"
Jul 13 19:06:32.082184 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.082101    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"
Jul 13 19:06:32.179771 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.179698    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"]
Jul 13 19:06:32.183424 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.183401    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:32.186846 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.186774    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xfcfp\""
Jul 13 19:06:32.187009 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.186873    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-991a1835-serving-cert\""
Jul 13 19:06:32.187425 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.187376    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 13 19:06:32.198335 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.198300    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"]
Jul 13 19:06:32.240474 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.240450    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"]
Jul 13 19:06:32.242057 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.242030    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38771ac2-0b3e-4d28-97d6-eb16b80da538-maas-api-tls\") pod \"maas-api-e2e-mt-991a1835-844d64648c-l4khh\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:32.242180 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.242118    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqkgj\" (UniqueName: \"kubernetes.io/projected/38771ac2-0b3e-4d28-97d6-eb16b80da538-kube-api-access-lqkgj\") pod \"maas-api-e2e-mt-991a1835-844d64648c-l4khh\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:32.242237 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.242182    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38771ac2-0b3e-4d28-97d6-eb16b80da538-openshift-service-ca\") pod \"maas-api-e2e-mt-991a1835-844d64648c-l4khh\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:32.242421 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:06:32.242399    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff39b2ab_d7d8_4e50_b454_94f15dffa6a2.slice/crio-dd810e42acc5e6e8589ee10eb2786ff60da56313e786c58a842a200914532c2c WatchSource:0}: Error finding container dd810e42acc5e6e8589ee10eb2786ff60da56313e786c58a842a200914532c2c: Status 404 returned error can't find the container with id dd810e42acc5e6e8589ee10eb2786ff60da56313e786c58a842a200914532c2c
Jul 13 19:06:32.259710 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.259565    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp" event={"ID":"ff39b2ab-d7d8-4e50-b454-94f15dffa6a2","Type":"ContainerStarted","Data":"dd810e42acc5e6e8589ee10eb2786ff60da56313e786c58a842a200914532c2c"}
Jul 13 19:06:32.342927 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.342838    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqkgj\" (UniqueName: \"kubernetes.io/projected/38771ac2-0b3e-4d28-97d6-eb16b80da538-kube-api-access-lqkgj\") pod \"maas-api-e2e-mt-991a1835-844d64648c-l4khh\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:32.342927 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.342907    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38771ac2-0b3e-4d28-97d6-eb16b80da538-openshift-service-ca\") pod \"maas-api-e2e-mt-991a1835-844d64648c-l4khh\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:32.343154 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.342942    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38771ac2-0b3e-4d28-97d6-eb16b80da538-maas-api-tls\") pod \"maas-api-e2e-mt-991a1835-844d64648c-l4khh\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:32.343154 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:06:32.343073    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-serving-cert: secret "maas-api-e2e-mt-991a1835-serving-cert" not found
Jul 13 19:06:32.343154 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:06:32.343142    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38771ac2-0b3e-4d28-97d6-eb16b80da538-maas-api-tls podName:38771ac2-0b3e-4d28-97d6-eb16b80da538 nodeName:}" failed. No retries permitted until 2026-07-13 19:06:32.84312232 +0000 UTC m=+2073.974410758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/38771ac2-0b3e-4d28-97d6-eb16b80da538-maas-api-tls") pod "maas-api-e2e-mt-991a1835-844d64648c-l4khh" (UID: "38771ac2-0b3e-4d28-97d6-eb16b80da538") : secret "maas-api-e2e-mt-991a1835-serving-cert" not found
Jul 13 19:06:32.343627 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.343602    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38771ac2-0b3e-4d28-97d6-eb16b80da538-openshift-service-ca\") pod \"maas-api-e2e-mt-991a1835-844d64648c-l4khh\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:32.354214 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.354185    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqkgj\" (UniqueName: \"kubernetes.io/projected/38771ac2-0b3e-4d28-97d6-eb16b80da538-kube-api-access-lqkgj\") pod \"maas-api-e2e-mt-991a1835-844d64648c-l4khh\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:32.848128 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.848079    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38771ac2-0b3e-4d28-97d6-eb16b80da538-maas-api-tls\") pod \"maas-api-e2e-mt-991a1835-844d64648c-l4khh\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:32.850689 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:32.850646    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38771ac2-0b3e-4d28-97d6-eb16b80da538-maas-api-tls\") pod \"maas-api-e2e-mt-991a1835-844d64648c-l4khh\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:33.096240 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:33.096210    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:33.233991 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:33.233967    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"]
Jul 13 19:06:33.236030 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:06:33.236005    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38771ac2_0b3e_4d28_97d6_eb16b80da538.slice/crio-59c0c48cf4ef7f1cc8b60e6c047311c442d2fe7cda1bb313f576ffb3fe72b0c5 WatchSource:0}: Error finding container 59c0c48cf4ef7f1cc8b60e6c047311c442d2fe7cda1bb313f576ffb3fe72b0c5: Status 404 returned error can't find the container with id 59c0c48cf4ef7f1cc8b60e6c047311c442d2fe7cda1bb313f576ffb3fe72b0c5
Jul 13 19:06:33.265303 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:33.265269    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh" event={"ID":"38771ac2-0b3e-4d28-97d6-eb16b80da538","Type":"ContainerStarted","Data":"59c0c48cf4ef7f1cc8b60e6c047311c442d2fe7cda1bb313f576ffb3fe72b0c5"}
Jul 13 19:06:33.266689 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:33.266635    2569 generic.go:356] "Generic (PLEG): container finished" podID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerID="fda10526cc2dfe5d9937545afff48ec15ed0162e5210a305b58548fe251598a0" exitCode=6
Jul 13 19:06:33.266785 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:33.266696    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp" event={"ID":"ff39b2ab-d7d8-4e50-b454-94f15dffa6a2","Type":"ContainerDied","Data":"fda10526cc2dfe5d9937545afff48ec15ed0162e5210a305b58548fe251598a0"}
Jul 13 19:06:33.266947 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:33.266934    2569 scope.go:117] "RemoveContainer" containerID="fda10526cc2dfe5d9937545afff48ec15ed0162e5210a305b58548fe251598a0"
Jul 13 19:06:34.273421 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:34.273375    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh" event={"ID":"38771ac2-0b3e-4d28-97d6-eb16b80da538","Type":"ContainerStarted","Data":"8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75"}
Jul 13 19:06:34.273927 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:34.273514    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:34.275255 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:34.275232    2569 generic.go:356] "Generic (PLEG): container finished" podID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerID="72f04f653e1c5217c1109b3f11fa9cb441009cc310555bc75ec0e6148452863a" exitCode=6
Jul 13 19:06:34.275346 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:34.275289    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp" event={"ID":"ff39b2ab-d7d8-4e50-b454-94f15dffa6a2","Type":"ContainerDied","Data":"72f04f653e1c5217c1109b3f11fa9cb441009cc310555bc75ec0e6148452863a"}
Jul 13 19:06:34.275346 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:34.275332    2569 scope.go:117] "RemoveContainer" containerID="fda10526cc2dfe5d9937545afff48ec15ed0162e5210a305b58548fe251598a0"
Jul 13 19:06:34.275504 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:34.275484    2569 scope.go:117] "RemoveContainer" containerID="72f04f653e1c5217c1109b3f11fa9cb441009cc310555bc75ec0e6148452863a"
Jul 13 19:06:34.275734 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:06:34.275714    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-mq5vp_odh-ai-gateway-infra(ff39b2ab-d7d8-4e50-b454-94f15dffa6a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2"
Jul 13 19:06:34.297621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:34.297577    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh" podStartSLOduration=1.790156095 podStartE2EDuration="2.297563017s" totalImagesPullingTime="507.406922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:06:34.295497237 +0000 UTC m=+2075.426785697" watchObservedRunningTime="2026-07-13 19:06:34.297563017 +0000 UTC m=+2075.428851513"
Jul 13 19:06:35.281550 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:35.281515    2569 scope.go:117] "RemoveContainer" containerID="72f04f653e1c5217c1109b3f11fa9cb441009cc310555bc75ec0e6148452863a"
Jul 13 19:06:35.281949 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:06:35.281750    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-mq5vp_odh-ai-gateway-infra(ff39b2ab-d7d8-4e50-b454-94f15dffa6a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2"
Jul 13 19:06:36.834917 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:36.834875    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:06:40.287121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:40.287089    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:06:46.344615 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:46.344590    2569 scope.go:117] "RemoveContainer" containerID="72f04f653e1c5217c1109b3f11fa9cb441009cc310555bc75ec0e6148452863a"
Jul 13 19:06:47.345180 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:47.345146    2569 generic.go:356] "Generic (PLEG): container finished" podID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerID="51c22f6ed5ee8ad8a5b21431fb140c893de8a9a759b91ea8a09abc11a0627e00" exitCode=6
Jul 13 19:06:47.349041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:47.349017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp" event={"ID":"ff39b2ab-d7d8-4e50-b454-94f15dffa6a2","Type":"ContainerDied","Data":"51c22f6ed5ee8ad8a5b21431fb140c893de8a9a759b91ea8a09abc11a0627e00"}
Jul 13 19:06:47.349164 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:47.349073    2569 scope.go:117] "RemoveContainer" containerID="72f04f653e1c5217c1109b3f11fa9cb441009cc310555bc75ec0e6148452863a"
Jul 13 19:06:47.349398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:47.349377    2569 scope.go:117] "RemoveContainer" containerID="51c22f6ed5ee8ad8a5b21431fb140c893de8a9a759b91ea8a09abc11a0627e00"
Jul 13 19:06:47.349590 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:06:47.349571    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-mq5vp_odh-ai-gateway-infra(ff39b2ab-d7d8-4e50-b454-94f15dffa6a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2"
Jul 13 19:06:48.380065 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:48.380032    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"]
Jul 13 19:06:48.512499 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:48.512474    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"
Jul 13 19:06:48.588119 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:48.588085    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff39b2ab-d7d8-4e50-b454-94f15dffa6a2-maas-api-service-ca\") pod \"ff39b2ab-d7d8-4e50-b454-94f15dffa6a2\" (UID: \"ff39b2ab-d7d8-4e50-b454-94f15dffa6a2\") "
Jul 13 19:06:48.588426 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:48.588404    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff39b2ab-d7d8-4e50-b454-94f15dffa6a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" (UID: "ff39b2ab-d7d8-4e50-b454-94f15dffa6a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:06:48.688706 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:48.688628    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff39b2ab-d7d8-4e50-b454-94f15dffa6a2-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:06:49.357373 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:49.357344    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"
Jul 13 19:06:49.357562 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:49.357381    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp" event={"ID":"ff39b2ab-d7d8-4e50-b454-94f15dffa6a2","Type":"ContainerDied","Data":"dd810e42acc5e6e8589ee10eb2786ff60da56313e786c58a842a200914532c2c"}
Jul 13 19:06:49.357562 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:49.357452    2569 scope.go:117] "RemoveContainer" containerID="51c22f6ed5ee8ad8a5b21431fb140c893de8a9a759b91ea8a09abc11a0627e00"
Jul 13 19:06:49.381836 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:49.381811    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"]
Jul 13 19:06:49.390879 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:49.390853    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-mq5vp"]
Jul 13 19:06:51.351051 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:51.351023    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" path="/var/lib/kubelet/pods/ff39b2ab-d7d8-4e50-b454-94f15dffa6a2/volumes"
Jul 13 19:06:55.756047 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.756018    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"]
Jul 13 19:06:55.756508 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.756353    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerName="revoke-keys"
Jul 13 19:06:55.756508 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.756364    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerName="revoke-keys"
Jul 13 19:06:55.756508 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.756374    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerName="revoke-keys"
Jul 13 19:06:55.756508 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.756380    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerName="revoke-keys"
Jul 13 19:06:55.756508 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.756394    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerName="revoke-keys"
Jul 13 19:06:55.756508 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.756399    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerName="revoke-keys"
Jul 13 19:06:55.756508 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.756451    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerName="revoke-keys"
Jul 13 19:06:55.756508 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.756457    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerName="revoke-keys"
Jul 13 19:06:55.756508 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.756464    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff39b2ab-d7d8-4e50-b454-94f15dffa6a2" containerName="revoke-keys"
Jul 13 19:06:55.761012 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.760991    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.764400 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.764375    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-991a1835\"/\"kube-root-ca.crt\""
Jul 13 19:06:55.764525 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.764452    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-991a1835\"/\"openshift-service-ca.crt\""
Jul 13 19:06:55.764525 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.764453    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-991a1835\"/\"default-dockercfg-wzzrl\""
Jul 13 19:06:55.764630 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.764456    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-991a1835\"/\"e2e-lifecycle-model-991a1835-kserve-self-signed-certs\""
Jul 13 19:06:55.772596 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.772572    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"]
Jul 13 19:06:55.850427 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.850394    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-home\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.850621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.850447    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rzc5\" (UniqueName: \"kubernetes.io/projected/ff9dc805-6701-4a80-9026-44a102c1184b-kube-api-access-9rzc5\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.850621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.850478    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-dshm\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.850621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.850510    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-tmp-dir\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.850621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.850580    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-model-cache\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.850827 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.850644    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9dc805-6701-4a80-9026-44a102c1184b-tls-certs\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.951240 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.951208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-home\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.951240 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.951252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rzc5\" (UniqueName: \"kubernetes.io/projected/ff9dc805-6701-4a80-9026-44a102c1184b-kube-api-access-9rzc5\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.951488 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.951280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-dshm\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.951488 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.951303    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-tmp-dir\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.951488 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.951335    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-model-cache\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.951488 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.951387    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9dc805-6701-4a80-9026-44a102c1184b-tls-certs\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.951720 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.951643    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-home\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.951823 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.951802    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-model-cache\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.951911 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.951893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-tmp-dir\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.953959 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.953924    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-dshm\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.954070 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.953972    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9dc805-6701-4a80-9026-44a102c1184b-tls-certs\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:55.959777 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:55.959755    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rzc5\" (UniqueName: \"kubernetes.io/projected/ff9dc805-6701-4a80-9026-44a102c1184b-kube-api-access-9rzc5\") pod \"e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") " pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:56.069928 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:56.069846    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:06:56.210628 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:56.210602    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"]
Jul 13 19:06:56.212834 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:06:56.212795    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff9dc805_6701_4a80_9026_44a102c1184b.slice/crio-5576964031d6f6353cc3d2518716ce32f85c999bca9c2b82652638b68029d44c WatchSource:0}: Error finding container 5576964031d6f6353cc3d2518716ce32f85c999bca9c2b82652638b68029d44c: Status 404 returned error can't find the container with id 5576964031d6f6353cc3d2518716ce32f85c999bca9c2b82652638b68029d44c
Jul 13 19:06:56.394674 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:56.394570    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv" event={"ID":"ff9dc805-6701-4a80-9026-44a102c1184b","Type":"ContainerStarted","Data":"c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046"}
Jul 13 19:06:56.394674 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:56.394620    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv" event={"ID":"ff9dc805-6701-4a80-9026-44a102c1184b","Type":"ContainerStarted","Data":"5576964031d6f6353cc3d2518716ce32f85c999bca9c2b82652638b68029d44c"}
Jul 13 19:06:56.414604 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:56.414545    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv" podStartSLOduration=1.4145282049999999 podStartE2EDuration="1.414528205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:06:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:06:56.411752499 +0000 UTC m=+2097.543040958" watchObservedRunningTime="2026-07-13 19:06:56.414528205 +0000 UTC m=+2097.545816664"
Jul 13 19:06:58.842156 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:58.842118    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:06:59.427904 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:59.427879    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 19:06:59.433298 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:06:59.433277    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 19:07:06.070817 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:06.070780    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:07:06.071198 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:06.070829    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:07:06.083455 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:06.083431    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:07:06.458082 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:06.458002    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:07:21.814680 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:21.814617    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"]
Jul 13 19:07:21.818270 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:21.818248    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"
Jul 13 19:07:21.820747 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:21.820725    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:07:21.825895 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:21.825863    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"]
Jul 13 19:07:21.868167 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:21.868130    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e299b5ef-957a-4418-a75b-c68a9289b02c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-7678h\" (UID: \"e299b5ef-957a-4418-a75b-c68a9289b02c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"
Jul 13 19:07:21.943436 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:21.943398    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:07:21.968855 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:21.968827    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e299b5ef-957a-4418-a75b-c68a9289b02c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-7678h\" (UID: \"e299b5ef-957a-4418-a75b-c68a9289b02c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"
Jul 13 19:07:21.969463 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:21.969446    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e299b5ef-957a-4418-a75b-c68a9289b02c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-7678h\" (UID: \"e299b5ef-957a-4418-a75b-c68a9289b02c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"
Jul 13 19:07:22.129997 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:22.129908    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"
Jul 13 19:07:22.471591 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:22.471568    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"]
Jul 13 19:07:22.473049 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:07:22.473021    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode299b5ef_957a_4418_a75b_c68a9289b02c.slice/crio-b16a064503f32aa0c697b55a6f1a77fa2357e8c88992db36939d802c282f38e4 WatchSource:0}: Error finding container b16a064503f32aa0c697b55a6f1a77fa2357e8c88992db36939d802c282f38e4: Status 404 returned error can't find the container with id b16a064503f32aa0c697b55a6f1a77fa2357e8c88992db36939d802c282f38e4
Jul 13 19:07:22.532944 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:22.532915    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h" event={"ID":"e299b5ef-957a-4418-a75b-c68a9289b02c","Type":"ContainerStarted","Data":"b16a064503f32aa0c697b55a6f1a77fa2357e8c88992db36939d802c282f38e4"}
Jul 13 19:07:23.539603 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:23.539567    2569 generic.go:356] "Generic (PLEG): container finished" podID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerID="9dce2fa9c732c22e055993c003fa5ba04365feb71b18a1122a9a32982dd76104" exitCode=6
Jul 13 19:07:23.540048 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:23.539679    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h" event={"ID":"e299b5ef-957a-4418-a75b-c68a9289b02c","Type":"ContainerDied","Data":"9dce2fa9c732c22e055993c003fa5ba04365feb71b18a1122a9a32982dd76104"}
Jul 13 19:07:23.540048 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:23.539827    2569 scope.go:117] "RemoveContainer" containerID="9dce2fa9c732c22e055993c003fa5ba04365feb71b18a1122a9a32982dd76104"
Jul 13 19:07:24.547541 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:24.547497    2569 generic.go:356] "Generic (PLEG): container finished" podID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerID="bc521cd215b39c5491dae6bde6a01b3fcef17f2839ba3b14a9ed36bc5602f122" exitCode=6
Jul 13 19:07:24.548000 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:24.547564    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h" event={"ID":"e299b5ef-957a-4418-a75b-c68a9289b02c","Type":"ContainerDied","Data":"bc521cd215b39c5491dae6bde6a01b3fcef17f2839ba3b14a9ed36bc5602f122"}
Jul 13 19:07:24.548000 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:24.547636    2569 scope.go:117] "RemoveContainer" containerID="9dce2fa9c732c22e055993c003fa5ba04365feb71b18a1122a9a32982dd76104"
Jul 13 19:07:24.548000 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:24.547878    2569 scope.go:117] "RemoveContainer" containerID="bc521cd215b39c5491dae6bde6a01b3fcef17f2839ba3b14a9ed36bc5602f122"
Jul 13 19:07:24.548141 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:07:24.548123    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-7678h_odh-ai-gateway-infra(e299b5ef-957a-4418-a75b-c68a9289b02c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c"
Jul 13 19:07:25.173302 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:25.173259    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj"]
Jul 13 19:07:25.177883 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:25.177860    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj"
Jul 13 19:07:25.195369 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:25.195343    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj"]
Jul 13 19:07:25.196188 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:25.196167    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18adf4b-9dd1-4df9-840c-46e1fb812492-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-991a1835-fltkj\" (UID: \"a18adf4b-9dd1-4df9-840c-46e1fb812492\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj"
Jul 13 19:07:25.297027 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:25.296998    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18adf4b-9dd1-4df9-840c-46e1fb812492-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-991a1835-fltkj\" (UID: \"a18adf4b-9dd1-4df9-840c-46e1fb812492\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj"
Jul 13 19:07:25.297631 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:25.297607    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18adf4b-9dd1-4df9-840c-46e1fb812492-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-991a1835-fltkj\" (UID: \"a18adf4b-9dd1-4df9-840c-46e1fb812492\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj"
Jul 13 19:07:25.487504 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:25.487473    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj"
Jul 13 19:07:25.559928 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:25.559897    2569 scope.go:117] "RemoveContainer" containerID="bc521cd215b39c5491dae6bde6a01b3fcef17f2839ba3b14a9ed36bc5602f122"
Jul 13 19:07:25.560328 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:07:25.560164    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-7678h_odh-ai-gateway-infra(e299b5ef-957a-4418-a75b-c68a9289b02c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c"
Jul 13 19:07:25.627637 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:25.627613    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj"]
Jul 13 19:07:25.629820 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:07:25.629791    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18adf4b_9dd1_4df9_840c_46e1fb812492.slice/crio-e1efd13ac6e2ca3e17491148f3d080a5579bf3460d29b554c9bf564e0ff842e6 WatchSource:0}: Error finding container e1efd13ac6e2ca3e17491148f3d080a5579bf3460d29b554c9bf564e0ff842e6: Status 404 returned error can't find the container with id e1efd13ac6e2ca3e17491148f3d080a5579bf3460d29b554c9bf564e0ff842e6
Jul 13 19:07:26.566335 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:26.566303    2569 generic.go:356] "Generic (PLEG): container finished" podID="a18adf4b-9dd1-4df9-840c-46e1fb812492" containerID="5918973daa32a26c07daa32b88a6d7a8a249b5548813e5e4a6f5ce66e04f7b8f" exitCode=0
Jul 13 19:07:26.566769 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:26.566382    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj" event={"ID":"a18adf4b-9dd1-4df9-840c-46e1fb812492","Type":"ContainerDied","Data":"5918973daa32a26c07daa32b88a6d7a8a249b5548813e5e4a6f5ce66e04f7b8f"}
Jul 13 19:07:26.566769 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:26.566432    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj" event={"ID":"a18adf4b-9dd1-4df9-840c-46e1fb812492","Type":"ContainerStarted","Data":"e1efd13ac6e2ca3e17491148f3d080a5579bf3460d29b554c9bf564e0ff842e6"}
Jul 13 19:07:27.712376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:27.712351    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj"
Jul 13 19:07:27.822089 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:27.821989    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18adf4b-9dd1-4df9-840c-46e1fb812492-maas-api-service-ca\") pod \"a18adf4b-9dd1-4df9-840c-46e1fb812492\" (UID: \"a18adf4b-9dd1-4df9-840c-46e1fb812492\") "
Jul 13 19:07:27.822353 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:27.822330    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a18adf4b-9dd1-4df9-840c-46e1fb812492-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a18adf4b-9dd1-4df9-840c-46e1fb812492" (UID: "a18adf4b-9dd1-4df9-840c-46e1fb812492"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:07:27.923223 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:27.923188    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18adf4b-9dd1-4df9-840c-46e1fb812492-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:07:28.577921 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:28.577894    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj"
Jul 13 19:07:28.578100 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:28.577920    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-991a1835-fltkj" event={"ID":"a18adf4b-9dd1-4df9-840c-46e1fb812492","Type":"ContainerDied","Data":"e1efd13ac6e2ca3e17491148f3d080a5579bf3460d29b554c9bf564e0ff842e6"}
Jul 13 19:07:28.578100 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:28.577959    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1efd13ac6e2ca3e17491148f3d080a5579bf3460d29b554c9bf564e0ff842e6"
Jul 13 19:07:35.664111 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:35.664047    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"]
Jul 13 19:07:35.664614 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:35.664328    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh" podUID="38771ac2-0b3e-4d28-97d6-eb16b80da538" containerName="maas-api" containerID="cri-o://8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75" gracePeriod=30
Jul 13 19:07:35.916980 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:35.916918    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:07:35.999228 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:35.999184    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38771ac2-0b3e-4d28-97d6-eb16b80da538-openshift-service-ca\") pod \"38771ac2-0b3e-4d28-97d6-eb16b80da538\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") "
Jul 13 19:07:35.999419 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:35.999234    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38771ac2-0b3e-4d28-97d6-eb16b80da538-maas-api-tls\") pod \"38771ac2-0b3e-4d28-97d6-eb16b80da538\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") "
Jul 13 19:07:35.999419 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:35.999279    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqkgj\" (UniqueName: \"kubernetes.io/projected/38771ac2-0b3e-4d28-97d6-eb16b80da538-kube-api-access-lqkgj\") pod \"38771ac2-0b3e-4d28-97d6-eb16b80da538\" (UID: \"38771ac2-0b3e-4d28-97d6-eb16b80da538\") "
Jul 13 19:07:35.999623 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:35.999595    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38771ac2-0b3e-4d28-97d6-eb16b80da538-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "38771ac2-0b3e-4d28-97d6-eb16b80da538" (UID: "38771ac2-0b3e-4d28-97d6-eb16b80da538"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:07:36.001568 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.001540    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38771ac2-0b3e-4d28-97d6-eb16b80da538-kube-api-access-lqkgj" (OuterVolumeSpecName: "kube-api-access-lqkgj") pod "38771ac2-0b3e-4d28-97d6-eb16b80da538" (UID: "38771ac2-0b3e-4d28-97d6-eb16b80da538"). InnerVolumeSpecName "kube-api-access-lqkgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:07:36.001717 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.001602    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38771ac2-0b3e-4d28-97d6-eb16b80da538-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "38771ac2-0b3e-4d28-97d6-eb16b80da538" (UID: "38771ac2-0b3e-4d28-97d6-eb16b80da538"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:07:36.100674 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.100622    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38771ac2-0b3e-4d28-97d6-eb16b80da538-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:07:36.100674 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.100677    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38771ac2-0b3e-4d28-97d6-eb16b80da538-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:07:36.100882 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.100692    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqkgj\" (UniqueName: \"kubernetes.io/projected/38771ac2-0b3e-4d28-97d6-eb16b80da538-kube-api-access-lqkgj\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:07:36.618147 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.618108    2569 generic.go:356] "Generic (PLEG): container finished" podID="38771ac2-0b3e-4d28-97d6-eb16b80da538" containerID="8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75" exitCode=0
Jul 13 19:07:36.618329 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.618163    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"
Jul 13 19:07:36.618329 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.618179    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh" event={"ID":"38771ac2-0b3e-4d28-97d6-eb16b80da538","Type":"ContainerDied","Data":"8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75"}
Jul 13 19:07:36.618329 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.618233    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh" event={"ID":"38771ac2-0b3e-4d28-97d6-eb16b80da538","Type":"ContainerDied","Data":"59c0c48cf4ef7f1cc8b60e6c047311c442d2fe7cda1bb313f576ffb3fe72b0c5"}
Jul 13 19:07:36.618329 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.618264    2569 scope.go:117] "RemoveContainer" containerID="8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75"
Jul 13 19:07:36.627679 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.627634    2569 scope.go:117] "RemoveContainer" containerID="8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75"
Jul 13 19:07:36.627936 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:07:36.627920    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75\": container with ID starting with 8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75 not found: ID does not exist" containerID="8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75"
Jul 13 19:07:36.627988 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.627944    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75"} err="failed to get container status \"8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75\": rpc error: code = NotFound desc = could not find container \"8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75\": container with ID starting with 8cd94c997ed2ab9db0454ac03f95f14c2e91174abab6b9378f2f43bae8990a75 not found: ID does not exist"
Jul 13 19:07:36.640705 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.640679    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"]
Jul 13 19:07:36.645172 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:36.645152    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-991a1835-844d64648c-l4khh"]
Jul 13 19:07:37.350449 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:37.350415    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38771ac2-0b3e-4d28-97d6-eb16b80da538" path="/var/lib/kubelet/pods/38771ac2-0b3e-4d28-97d6-eb16b80da538/volumes"
Jul 13 19:07:37.351018 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:37.351004    2569 scope.go:117] "RemoveContainer" containerID="bc521cd215b39c5491dae6bde6a01b3fcef17f2839ba3b14a9ed36bc5602f122"
Jul 13 19:07:37.625700 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:37.625667    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h" event={"ID":"e299b5ef-957a-4418-a75b-c68a9289b02c","Type":"ContainerStarted","Data":"04a6ab6ec3047626419a59c3f5d044f2a698e0d663e444d85ec4da30a394e6aa"}
Jul 13 19:07:37.644345 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:37.644305    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h" podStartSLOduration=16.644286094 podStartE2EDuration="16.644286094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:07:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:07:37.642292739 +0000 UTC m=+2138.773581196" watchObservedRunningTime="2026-07-13 19:07:37.644286094 +0000 UTC m=+2138.775574585"
Jul 13 19:07:38.632694 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:38.632640    2569 generic.go:356] "Generic (PLEG): container finished" podID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerID="04a6ab6ec3047626419a59c3f5d044f2a698e0d663e444d85ec4da30a394e6aa" exitCode=6
Jul 13 19:07:38.633128 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:38.632716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h" event={"ID":"e299b5ef-957a-4418-a75b-c68a9289b02c","Type":"ContainerDied","Data":"04a6ab6ec3047626419a59c3f5d044f2a698e0d663e444d85ec4da30a394e6aa"}
Jul 13 19:07:38.633128 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:38.632774    2569 scope.go:117] "RemoveContainer" containerID="bc521cd215b39c5491dae6bde6a01b3fcef17f2839ba3b14a9ed36bc5602f122"
Jul 13 19:07:38.633128 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:38.633042    2569 scope.go:117] "RemoveContainer" containerID="04a6ab6ec3047626419a59c3f5d044f2a698e0d663e444d85ec4da30a394e6aa"
Jul 13 19:07:38.633264 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:07:38.633240    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-7678h_odh-ai-gateway-infra(e299b5ef-957a-4418-a75b-c68a9289b02c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c"
Jul 13 19:07:38.659782 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:38.659748    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"]
Jul 13 19:07:39.534600 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:39.534568    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:07:39.768335 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:39.768313    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"
Jul 13 19:07:39.832947 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:39.832855    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e299b5ef-957a-4418-a75b-c68a9289b02c-maas-api-service-ca\") pod \"e299b5ef-957a-4418-a75b-c68a9289b02c\" (UID: \"e299b5ef-957a-4418-a75b-c68a9289b02c\") "
Jul 13 19:07:39.833253 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:39.833229    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e299b5ef-957a-4418-a75b-c68a9289b02c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e299b5ef-957a-4418-a75b-c68a9289b02c" (UID: "e299b5ef-957a-4418-a75b-c68a9289b02c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:07:39.934462 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:39.934428    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e299b5ef-957a-4418-a75b-c68a9289b02c-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:07:40.645770 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:40.645732    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h" event={"ID":"e299b5ef-957a-4418-a75b-c68a9289b02c","Type":"ContainerDied","Data":"b16a064503f32aa0c697b55a6f1a77fa2357e8c88992db36939d802c282f38e4"}
Jul 13 19:07:40.645770 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:40.645755    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"
Jul 13 19:07:40.646017 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:40.645806    2569 scope.go:117] "RemoveContainer" containerID="04a6ab6ec3047626419a59c3f5d044f2a698e0d663e444d85ec4da30a394e6aa"
Jul 13 19:07:40.672573 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:40.672544    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"]
Jul 13 19:07:40.678556 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:40.678530    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-7678h"]
Jul 13 19:07:41.350981 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:41.350946    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c" path="/var/lib/kubelet/pods/e299b5ef-957a-4418-a75b-c68a9289b02c/volumes"
Jul 13 19:07:45.226541 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:07:45.226509    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"]
Jul 13 19:08:00.247233 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.247164    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv" podUID="ff9dc805-6701-4a80-9026-44a102c1184b" containerName="main" containerID="cri-o://c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046" gracePeriod=45
Jul 13 19:08:00.508373 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.508317    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:08:00.611991 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.611959    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-dshm\") pod \"ff9dc805-6701-4a80-9026-44a102c1184b\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") "
Jul 13 19:08:00.612181 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612015    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-model-cache\") pod \"ff9dc805-6701-4a80-9026-44a102c1184b\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") "
Jul 13 19:08:00.612181 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612057    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-home\") pod \"ff9dc805-6701-4a80-9026-44a102c1184b\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") "
Jul 13 19:08:00.612181 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612123    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-tmp-dir\") pod \"ff9dc805-6701-4a80-9026-44a102c1184b\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") "
Jul 13 19:08:00.612296 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612180    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rzc5\" (UniqueName: \"kubernetes.io/projected/ff9dc805-6701-4a80-9026-44a102c1184b-kube-api-access-9rzc5\") pod \"ff9dc805-6701-4a80-9026-44a102c1184b\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") "
Jul 13 19:08:00.612296 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612210    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9dc805-6701-4a80-9026-44a102c1184b-tls-certs\") pod \"ff9dc805-6701-4a80-9026-44a102c1184b\" (UID: \"ff9dc805-6701-4a80-9026-44a102c1184b\") "
Jul 13 19:08:00.612381 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612306    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff9dc805-6701-4a80-9026-44a102c1184b" (UID: "ff9dc805-6701-4a80-9026-44a102c1184b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:08:00.612421 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612387    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-home" (OuterVolumeSpecName: "home") pod "ff9dc805-6701-4a80-9026-44a102c1184b" (UID: "ff9dc805-6701-4a80-9026-44a102c1184b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:08:00.612421 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612396    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff9dc805-6701-4a80-9026-44a102c1184b" (UID: "ff9dc805-6701-4a80-9026-44a102c1184b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:08:00.612548 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612524    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-home\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:08:00.612596 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612550    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-tmp-dir\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:08:00.612596 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.612565    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-model-cache\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:08:00.614483 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.614448    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-dshm" (OuterVolumeSpecName: "dshm") pod "ff9dc805-6701-4a80-9026-44a102c1184b" (UID: "ff9dc805-6701-4a80-9026-44a102c1184b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:08:00.614589 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.614503    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9dc805-6701-4a80-9026-44a102c1184b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff9dc805-6701-4a80-9026-44a102c1184b" (UID: "ff9dc805-6701-4a80-9026-44a102c1184b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:08:00.614589 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.614528    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9dc805-6701-4a80-9026-44a102c1184b-kube-api-access-9rzc5" (OuterVolumeSpecName: "kube-api-access-9rzc5") pod "ff9dc805-6701-4a80-9026-44a102c1184b" (UID: "ff9dc805-6701-4a80-9026-44a102c1184b"). InnerVolumeSpecName "kube-api-access-9rzc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:08:00.713797 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.713766    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9dc805-6701-4a80-9026-44a102c1184b-dshm\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:08:00.713797 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.713792    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rzc5\" (UniqueName: \"kubernetes.io/projected/ff9dc805-6701-4a80-9026-44a102c1184b-kube-api-access-9rzc5\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:08:00.713797 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.713803    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9dc805-6701-4a80-9026-44a102c1184b-tls-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:08:00.751317 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.751282    2569 generic.go:356] "Generic (PLEG): container finished" podID="ff9dc805-6701-4a80-9026-44a102c1184b" containerID="c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046" exitCode=0
Jul 13 19:08:00.751444 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.751328    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"
Jul 13 19:08:00.751444 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.751335    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv" event={"ID":"ff9dc805-6701-4a80-9026-44a102c1184b","Type":"ContainerDied","Data":"c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046"}
Jul 13 19:08:00.751444 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.751374    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv" event={"ID":"ff9dc805-6701-4a80-9026-44a102c1184b","Type":"ContainerDied","Data":"5576964031d6f6353cc3d2518716ce32f85c999bca9c2b82652638b68029d44c"}
Jul 13 19:08:00.751444 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.751395    2569 scope.go:117] "RemoveContainer" containerID="c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046"
Jul 13 19:08:00.760472 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.760454    2569 scope.go:117] "RemoveContainer" containerID="c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046"
Jul 13 19:08:00.760794 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:08:00.760775    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046\": container with ID starting with c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046 not found: ID does not exist" containerID="c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046"
Jul 13 19:08:00.760877 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.760808    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046"} err="failed to get container status \"c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046\": rpc error: code = NotFound desc = could not find container \"c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046\": container with ID starting with c328ff5b617ce6f727e06c302ff52825596bac82edd019f622d3dbfdd1ede046 not found: ID does not exist"
Jul 13 19:08:00.772737 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.772715    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"]
Jul 13 19:08:00.778831 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:00.778810    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-991a1835/e2e-lifecycle-model-991a1835-kserve-d59c64c79-4t5xv"]
Jul 13 19:08:01.350906 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:01.350867    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9dc805-6701-4a80-9026-44a102c1184b" path="/var/lib/kubelet/pods/ff9dc805-6701-4a80-9026-44a102c1184b/volumes"
Jul 13 19:08:11.861994 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.861959    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"]
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862275    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9dc805-6701-4a80-9026-44a102c1184b" containerName="main"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862285    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9dc805-6701-4a80-9026-44a102c1184b" containerName="main"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862295    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerName="revoke-keys"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862300    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerName="revoke-keys"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862313    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38771ac2-0b3e-4d28-97d6-eb16b80da538" containerName="maas-api"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862318    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="38771ac2-0b3e-4d28-97d6-eb16b80da538" containerName="maas-api"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862327    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerName="revoke-keys"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862332    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerName="revoke-keys"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862340    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18adf4b-9dd1-4df9-840c-46e1fb812492" containerName="revoke-keys"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862345    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18adf4b-9dd1-4df9-840c-46e1fb812492" containerName="revoke-keys"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862351    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerName="revoke-keys"
Jul 13 19:08:11.862398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862358    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerName="revoke-keys"
Jul 13 19:08:11.862890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862410    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerName="revoke-keys"
Jul 13 19:08:11.862890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862419    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerName="revoke-keys"
Jul 13 19:08:11.862890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862425    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9dc805-6701-4a80-9026-44a102c1184b" containerName="main"
Jul 13 19:08:11.862890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862432    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="38771ac2-0b3e-4d28-97d6-eb16b80da538" containerName="maas-api"
Jul 13 19:08:11.862890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.862439    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18adf4b-9dd1-4df9-840c-46e1fb812492" containerName="revoke-keys"
Jul 13 19:08:11.865440 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.865422    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"
Jul 13 19:08:11.867921 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.867898    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:08:11.868052 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.868032    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:08:11.873808 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.873786    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"]
Jul 13 19:08:11.899610 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:11.899584    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-lp6np\" (UID: \"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"
Jul 13 19:08:12.000540 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:12.000500    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-lp6np\" (UID: \"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"
Jul 13 19:08:12.001190 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:12.001171    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-lp6np\" (UID: \"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"
Jul 13 19:08:12.176993 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:12.176893    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"
Jul 13 19:08:12.312565 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:12.312539    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"]
Jul 13 19:08:12.314813 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:08:12.314779    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51dc8df2_1fbe_4ab2_ad4b_2a9c7c37e654.slice/crio-e9ce4c6856848e957a35a31bc921f20b036b2cce32052e062c1192f194e7b3a7 WatchSource:0}: Error finding container e9ce4c6856848e957a35a31bc921f20b036b2cce32052e062c1192f194e7b3a7: Status 404 returned error can't find the container with id e9ce4c6856848e957a35a31bc921f20b036b2cce32052e062c1192f194e7b3a7
Jul 13 19:08:12.812709 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:12.812600    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" event={"ID":"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654","Type":"ContainerStarted","Data":"692a6f071ea8b49d932798edc308b8498bccc2a3b121fbed5dea60d2aa1d6d4c"}
Jul 13 19:08:12.812709 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:12.812664    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" event={"ID":"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654","Type":"ContainerStarted","Data":"e9ce4c6856848e957a35a31bc921f20b036b2cce32052e062c1192f194e7b3a7"}
Jul 13 19:08:12.829793 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:12.829739    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" podStartSLOduration=1.8297199389999999 podStartE2EDuration="1.829719939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:08:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:08:12.828741911 +0000 UTC m=+2173.960030372" watchObservedRunningTime="2026-07-13 19:08:12.829719939 +0000 UTC m=+2173.961008399"
Jul 13 19:08:13.818909 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:13.818872    2569 generic.go:356] "Generic (PLEG): container finished" podID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerID="692a6f071ea8b49d932798edc308b8498bccc2a3b121fbed5dea60d2aa1d6d4c" exitCode=6
Jul 13 19:08:13.819297 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:13.818969    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" event={"ID":"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654","Type":"ContainerDied","Data":"692a6f071ea8b49d932798edc308b8498bccc2a3b121fbed5dea60d2aa1d6d4c"}
Jul 13 19:08:13.819297 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:13.819109    2569 scope.go:117] "RemoveContainer" containerID="692a6f071ea8b49d932798edc308b8498bccc2a3b121fbed5dea60d2aa1d6d4c"
Jul 13 19:08:14.825977 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:14.825939    2569 generic.go:356] "Generic (PLEG): container finished" podID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerID="37a76dcc1baabb08ce6233fcd7415427ef4dfd88f9e7471e2735c8096a302832" exitCode=6
Jul 13 19:08:14.826407 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:14.826010    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" event={"ID":"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654","Type":"ContainerDied","Data":"37a76dcc1baabb08ce6233fcd7415427ef4dfd88f9e7471e2735c8096a302832"}
Jul 13 19:08:14.826407 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:14.826064    2569 scope.go:117] "RemoveContainer" containerID="692a6f071ea8b49d932798edc308b8498bccc2a3b121fbed5dea60d2aa1d6d4c"
Jul 13 19:08:14.826407 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:14.826261    2569 scope.go:117] "RemoveContainer" containerID="37a76dcc1baabb08ce6233fcd7415427ef4dfd88f9e7471e2735c8096a302832"
Jul 13 19:08:14.826555 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:08:14.826445    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-lp6np_odh-ai-gateway-infra(51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654"
Jul 13 19:08:15.832410 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:15.832378    2569 scope.go:117] "RemoveContainer" containerID="37a76dcc1baabb08ce6233fcd7415427ef4dfd88f9e7471e2735c8096a302832"
Jul 13 19:08:15.832831 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:08:15.832553    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-lp6np_odh-ai-gateway-infra(51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654"
Jul 13 19:08:29.347338 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:29.347310    2569 scope.go:117] "RemoveContainer" containerID="37a76dcc1baabb08ce6233fcd7415427ef4dfd88f9e7471e2735c8096a302832"
Jul 13 19:08:29.904748 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:29.904715    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" event={"ID":"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654","Type":"ContainerStarted","Data":"af8a504d50445b7afd7e5a15348b057b8d4821914289839836e4154ba377fc4f"}
Jul 13 19:08:30.911342 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:30.911308    2569 generic.go:356] "Generic (PLEG): container finished" podID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerID="af8a504d50445b7afd7e5a15348b057b8d4821914289839836e4154ba377fc4f" exitCode=6
Jul 13 19:08:30.911883 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:30.911376    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" event={"ID":"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654","Type":"ContainerDied","Data":"af8a504d50445b7afd7e5a15348b057b8d4821914289839836e4154ba377fc4f"}
Jul 13 19:08:30.911883 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:30.911434    2569 scope.go:117] "RemoveContainer" containerID="37a76dcc1baabb08ce6233fcd7415427ef4dfd88f9e7471e2735c8096a302832"
Jul 13 19:08:30.911883 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:30.911728    2569 scope.go:117] "RemoveContainer" containerID="af8a504d50445b7afd7e5a15348b057b8d4821914289839836e4154ba377fc4f"
Jul 13 19:08:30.912044 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:08:30.911918    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-lp6np_odh-ai-gateway-infra(51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654"
Jul 13 19:08:30.939094 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:30.939071    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"]
Jul 13 19:08:32.048101 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:32.048069    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"
Jul 13 19:08:32.171537 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:32.171454    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654-maas-api-service-ca\") pod \"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654\" (UID: \"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654\") "
Jul 13 19:08:32.171853 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:32.171830    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" (UID: "51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:08:32.272712 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:32.272677    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:08:32.923662 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:32.923620    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"
Jul 13 19:08:32.923847 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:32.923667    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np" event={"ID":"51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654","Type":"ContainerDied","Data":"e9ce4c6856848e957a35a31bc921f20b036b2cce32052e062c1192f194e7b3a7"}
Jul 13 19:08:32.923847 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:32.923719    2569 scope.go:117] "RemoveContainer" containerID="af8a504d50445b7afd7e5a15348b057b8d4821914289839836e4154ba377fc4f"
Jul 13 19:08:32.944851 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:32.944826    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"]
Jul 13 19:08:32.952052 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:32.952033    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-lp6np"]
Jul 13 19:08:33.350549 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:33.350514    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" path="/var/lib/kubelet/pods/51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654/volumes"
Jul 13 19:08:36.639184 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:36.639147    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:08:48.329982 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:08:48.329945    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:09:11.918813 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.918777    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"]
Jul 13 19:09:11.919276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.919100    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerName="revoke-keys"
Jul 13 19:09:11.919276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.919111    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerName="revoke-keys"
Jul 13 19:09:11.919276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.919118    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerName="revoke-keys"
Jul 13 19:09:11.919276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.919123    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerName="revoke-keys"
Jul 13 19:09:11.919276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.919136    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerName="revoke-keys"
Jul 13 19:09:11.919276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.919142    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerName="revoke-keys"
Jul 13 19:09:11.919276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.919202    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e299b5ef-957a-4418-a75b-c68a9289b02c" containerName="revoke-keys"
Jul 13 19:09:11.919276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.919210    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerName="revoke-keys"
Jul 13 19:09:11.919276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.919215    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerName="revoke-keys"
Jul 13 19:09:11.919276 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.919221    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="51dc8df2-1fbe-4ab2-ad4b-2a9c7c37e654" containerName="revoke-keys"
Jul 13 19:09:11.922225 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.922209    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"
Jul 13 19:09:11.925065 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.925046    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:09:11.925585 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.925570    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:09:11.932545 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:11.932520    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"]
Jul 13 19:09:12.006814 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:12.006775    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9d3a4d6-d107-448d-9fbc-e8466dd87a2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-fvckp\" (UID: \"e9d3a4d6-d107-448d-9fbc-e8466dd87a2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"
Jul 13 19:09:12.108067 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:12.108027    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9d3a4d6-d107-448d-9fbc-e8466dd87a2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-fvckp\" (UID: \"e9d3a4d6-d107-448d-9fbc-e8466dd87a2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"
Jul 13 19:09:12.108733 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:12.108712    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9d3a4d6-d107-448d-9fbc-e8466dd87a2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-fvckp\" (UID: \"e9d3a4d6-d107-448d-9fbc-e8466dd87a2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"
Jul 13 19:09:12.233041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:12.233012    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"
Jul 13 19:09:12.366465 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:12.366444    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"]
Jul 13 19:09:12.368630 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:09:12.368596    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9d3a4d6_d107_448d_9fbc_e8466dd87a2f.slice/crio-4976d2c2fab8d1318ea819b5083ebb221aa1a65bc3be4915e9f04a5a572e2d3e WatchSource:0}: Error finding container 4976d2c2fab8d1318ea819b5083ebb221aa1a65bc3be4915e9f04a5a572e2d3e: Status 404 returned error can't find the container with id 4976d2c2fab8d1318ea819b5083ebb221aa1a65bc3be4915e9f04a5a572e2d3e
Jul 13 19:09:13.134308 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:13.134275    2569 generic.go:356] "Generic (PLEG): container finished" podID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerID="f96dcb69aec434d07320e5dca6cf99eb570d429ea07e4876236db4f0016c1057" exitCode=6
Jul 13 19:09:13.134720 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:13.134370    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp" event={"ID":"e9d3a4d6-d107-448d-9fbc-e8466dd87a2f","Type":"ContainerDied","Data":"f96dcb69aec434d07320e5dca6cf99eb570d429ea07e4876236db4f0016c1057"}
Jul 13 19:09:13.134720 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:13.134436    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp" event={"ID":"e9d3a4d6-d107-448d-9fbc-e8466dd87a2f","Type":"ContainerStarted","Data":"4976d2c2fab8d1318ea819b5083ebb221aa1a65bc3be4915e9f04a5a572e2d3e"}
Jul 13 19:09:13.134720 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:13.134552    2569 scope.go:117] "RemoveContainer" containerID="f96dcb69aec434d07320e5dca6cf99eb570d429ea07e4876236db4f0016c1057"
Jul 13 19:09:14.141252 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:14.141215    2569 generic.go:356] "Generic (PLEG): container finished" podID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerID="8238e764ef5b3abdf142083e40b430dad470a36fdcb9b54f5642b6126e3339c7" exitCode=6
Jul 13 19:09:14.141749 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:14.141303    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp" event={"ID":"e9d3a4d6-d107-448d-9fbc-e8466dd87a2f","Type":"ContainerDied","Data":"8238e764ef5b3abdf142083e40b430dad470a36fdcb9b54f5642b6126e3339c7"}
Jul 13 19:09:14.141749 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:14.141374    2569 scope.go:117] "RemoveContainer" containerID="f96dcb69aec434d07320e5dca6cf99eb570d429ea07e4876236db4f0016c1057"
Jul 13 19:09:14.141749 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:14.141597    2569 scope.go:117] "RemoveContainer" containerID="8238e764ef5b3abdf142083e40b430dad470a36fdcb9b54f5642b6126e3339c7"
Jul 13 19:09:14.141925 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:09:14.141843    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-fvckp_odh-ai-gateway-infra(e9d3a4d6-d107-448d-9fbc-e8466dd87a2f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f"
Jul 13 19:09:15.148515 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:15.148488    2569 scope.go:117] "RemoveContainer" containerID="8238e764ef5b3abdf142083e40b430dad470a36fdcb9b54f5642b6126e3339c7"
Jul 13 19:09:15.148918 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:09:15.148676    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-fvckp_odh-ai-gateway-infra(e9d3a4d6-d107-448d-9fbc-e8466dd87a2f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f"
Jul 13 19:09:27.345169 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:27.345138    2569 scope.go:117] "RemoveContainer" containerID="8238e764ef5b3abdf142083e40b430dad470a36fdcb9b54f5642b6126e3339c7"
Jul 13 19:09:28.214479 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:28.214444    2569 generic.go:356] "Generic (PLEG): container finished" podID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerID="9da2471abf53cf654bfb0221825ba2d3a8e1512c4dd80de060b6f19161e00fb1" exitCode=6
Jul 13 19:09:28.214640 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:28.214515    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp" event={"ID":"e9d3a4d6-d107-448d-9fbc-e8466dd87a2f","Type":"ContainerDied","Data":"9da2471abf53cf654bfb0221825ba2d3a8e1512c4dd80de060b6f19161e00fb1"}
Jul 13 19:09:28.214640 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:28.214604    2569 scope.go:117] "RemoveContainer" containerID="8238e764ef5b3abdf142083e40b430dad470a36fdcb9b54f5642b6126e3339c7"
Jul 13 19:09:28.214971 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:28.214954    2569 scope.go:117] "RemoveContainer" containerID="9da2471abf53cf654bfb0221825ba2d3a8e1512c4dd80de060b6f19161e00fb1"
Jul 13 19:09:28.215203 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:09:28.215163    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-fvckp_odh-ai-gateway-infra(e9d3a4d6-d107-448d-9fbc-e8466dd87a2f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f"
Jul 13 19:09:29.243193 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:29.243153    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"]
Jul 13 19:09:29.376565 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:29.376541    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"
Jul 13 19:09:29.447581 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:29.447546    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9d3a4d6-d107-448d-9fbc-e8466dd87a2f-maas-api-service-ca\") pod \"e9d3a4d6-d107-448d-9fbc-e8466dd87a2f\" (UID: \"e9d3a4d6-d107-448d-9fbc-e8466dd87a2f\") "
Jul 13 19:09:29.447952 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:29.447927    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9d3a4d6-d107-448d-9fbc-e8466dd87a2f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" (UID: "e9d3a4d6-d107-448d-9fbc-e8466dd87a2f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:09:29.548715 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:29.548593    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9d3a4d6-d107-448d-9fbc-e8466dd87a2f-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:09:30.227300 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:30.227269    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"
Jul 13 19:09:30.227478 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:30.227309    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp" event={"ID":"e9d3a4d6-d107-448d-9fbc-e8466dd87a2f","Type":"ContainerDied","Data":"4976d2c2fab8d1318ea819b5083ebb221aa1a65bc3be4915e9f04a5a572e2d3e"}
Jul 13 19:09:30.227478 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:30.227378    2569 scope.go:117] "RemoveContainer" containerID="9da2471abf53cf654bfb0221825ba2d3a8e1512c4dd80de060b6f19161e00fb1"
Jul 13 19:09:30.249411 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:30.249382    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"]
Jul 13 19:09:30.255293 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:30.255271    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-fvckp"]
Jul 13 19:09:31.350606 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.350572    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" path="/var/lib/kubelet/pods/e9d3a4d6-d107-448d-9fbc-e8466dd87a2f/volumes"
Jul 13 19:09:31.846497 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.846462    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"]
Jul 13 19:09:31.846830 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.846817    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerName="revoke-keys"
Jul 13 19:09:31.846893 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.846831    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerName="revoke-keys"
Jul 13 19:09:31.846893 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.846843    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerName="revoke-keys"
Jul 13 19:09:31.846893 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.846848    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerName="revoke-keys"
Jul 13 19:09:31.846989 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.846908    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerName="revoke-keys"
Jul 13 19:09:31.846989 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.846918    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerName="revoke-keys"
Jul 13 19:09:31.846989 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.846923    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerName="revoke-keys"
Jul 13 19:09:31.851182 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.851159    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:31.853788 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.853765    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-5d44da-serving-cert\""
Jul 13 19:09:31.853921 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.853814    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:09:31.854509 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.854494    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 13 19:09:31.854589 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.854552    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xfcfp\""
Jul 13 19:09:31.861640 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.861619    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"]
Jul 13 19:09:31.969426 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.969384    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg6vx\" (UniqueName: \"kubernetes.io/projected/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-kube-api-access-kg6vx\") pod \"maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:31.969596 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.969462    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:31.969596 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:31.969507    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-maas-api-tls\") pod \"maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:32.070281 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:32.070248    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-maas-api-tls\") pod \"maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:32.070496 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:32.070303    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg6vx\" (UniqueName: \"kubernetes.io/projected/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-kube-api-access-kg6vx\") pod \"maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:32.070496 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:32.070366    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:32.071037 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:32.071013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:32.073040 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:32.073013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-maas-api-tls\") pod \"maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:32.080066 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:32.080036    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg6vx\" (UniqueName: \"kubernetes.io/projected/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-kube-api-access-kg6vx\") pod \"maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:32.161641 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:32.161539    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:32.310378 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:32.310349    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"]
Jul 13 19:09:32.312858 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:09:32.312826    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeabb1183_0a0e_48e2_bc5c_249e384fc4f8.slice/crio-b2499c0804d34eb572450abd16f5f5e76cfa90d8d02899ebaf4993bcea486d42 WatchSource:0}: Error finding container b2499c0804d34eb572450abd16f5f5e76cfa90d8d02899ebaf4993bcea486d42: Status 404 returned error can't find the container with id b2499c0804d34eb572450abd16f5f5e76cfa90d8d02899ebaf4993bcea486d42
Jul 13 19:09:32.314629 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:32.314608    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 19:09:33.246360 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:33.246319    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g" event={"ID":"eabb1183-0a0e-48e2-bc5c-249e384fc4f8","Type":"ContainerStarted","Data":"c79c7b26dd635cc022e278bf99bbe7764d47a46c8902b677bc2d498b98506c4c"}
Jul 13 19:09:33.246777 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:33.246421    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g" event={"ID":"eabb1183-0a0e-48e2-bc5c-249e384fc4f8","Type":"ContainerStarted","Data":"b2499c0804d34eb572450abd16f5f5e76cfa90d8d02899ebaf4993bcea486d42"}
Jul 13 19:09:33.246777 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:33.246451    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:33.273321 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:33.273267    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g" podStartSLOduration=1.896549173 podStartE2EDuration="2.273250689s" totalImagesPullingTime="376.701516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:09:33.269783687 +0000 UTC m=+2254.401072164" watchObservedRunningTime="2026-07-13 19:09:33.273250689 +0000 UTC m=+2254.404539148"
Jul 13 19:09:39.257435 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:39.257406    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:09:39.839842 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:39.839811    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:09:40.050145 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.050116    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"]
Jul 13 19:09:40.050462 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.050450    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerName="revoke-keys"
Jul 13 19:09:40.050538 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.050463    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9d3a4d6-d107-448d-9fbc-e8466dd87a2f" containerName="revoke-keys"
Jul 13 19:09:40.053733 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.053714    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.057910 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.057886    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-0a46d9-serving-cert\""
Jul 13 19:09:40.074759 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.074736    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"]
Jul 13 19:09:40.140733 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.140622    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4cbb70-75bb-444d-895b-34d99d6e8f17-maas-api-tls\") pod \"maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.140733 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.140680    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4cbb70-75bb-444d-895b-34d99d6e8f17-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.140733 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.140733    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r8bj\" (UniqueName: \"kubernetes.io/projected/4d4cbb70-75bb-444d-895b-34d99d6e8f17-kube-api-access-8r8bj\") pod \"maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.241185 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.241150    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r8bj\" (UniqueName: \"kubernetes.io/projected/4d4cbb70-75bb-444d-895b-34d99d6e8f17-kube-api-access-8r8bj\") pod \"maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.241363 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.241259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4cbb70-75bb-444d-895b-34d99d6e8f17-maas-api-tls\") pod \"maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.241363 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.241301    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4cbb70-75bb-444d-895b-34d99d6e8f17-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.241439 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:09:40.241414    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-serving-cert: secret "maas-api-e2e-inf-b-0a46d9-serving-cert" not found
Jul 13 19:09:40.241516 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:09:40.241505    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d4cbb70-75bb-444d-895b-34d99d6e8f17-maas-api-tls podName:4d4cbb70-75bb-444d-895b-34d99d6e8f17 nodeName:}" failed. No retries permitted until 2026-07-13 19:09:40.741484729 +0000 UTC m=+2261.872773170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4d4cbb70-75bb-444d-895b-34d99d6e8f17-maas-api-tls") pod "maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx" (UID: "4d4cbb70-75bb-444d-895b-34d99d6e8f17") : secret "maas-api-e2e-inf-b-0a46d9-serving-cert" not found
Jul 13 19:09:40.242019 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.242002    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4cbb70-75bb-444d-895b-34d99d6e8f17-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.250455 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.250426    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r8bj\" (UniqueName: \"kubernetes.io/projected/4d4cbb70-75bb-444d-895b-34d99d6e8f17-kube-api-access-8r8bj\") pod \"maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.746486 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.746451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4cbb70-75bb-444d-895b-34d99d6e8f17-maas-api-tls\") pod \"maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.748987 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.748964    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4cbb70-75bb-444d-895b-34d99d6e8f17-maas-api-tls\") pod \"maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:40.964391 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:40.964352    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:41.105036 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:41.105012    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"]
Jul 13 19:09:41.107016 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:09:41.106990    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d4cbb70_75bb_444d_895b_34d99d6e8f17.slice/crio-33108f278fbcffeecfa40a46d9479b75eb3a45d213fc3529cd5fd10fd6903420 WatchSource:0}: Error finding container 33108f278fbcffeecfa40a46d9479b75eb3a45d213fc3529cd5fd10fd6903420: Status 404 returned error can't find the container with id 33108f278fbcffeecfa40a46d9479b75eb3a45d213fc3529cd5fd10fd6903420
Jul 13 19:09:41.292194 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:41.292106    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx" event={"ID":"4d4cbb70-75bb-444d-895b-34d99d6e8f17","Type":"ContainerStarted","Data":"33108f278fbcffeecfa40a46d9479b75eb3a45d213fc3529cd5fd10fd6903420"}
Jul 13 19:09:42.300541 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:42.300497    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx" event={"ID":"4d4cbb70-75bb-444d-895b-34d99d6e8f17","Type":"ContainerStarted","Data":"3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa"}
Jul 13 19:09:42.300971 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:42.300644    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:42.329795 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:42.329740    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx" podStartSLOduration=1.917784051 podStartE2EDuration="2.329720857s" totalImagesPullingTime="411.936806ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:09:42.327881505 +0000 UTC m=+2263.459169972" watchObservedRunningTime="2026-07-13 19:09:42.329720857 +0000 UTC m=+2263.461009317"
Jul 13 19:09:48.312594 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:48.312565    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:09:50.552046 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.552010    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"]
Jul 13 19:09:50.556174 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.556154    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.559032 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.559005    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5d44da\"/\"default-dockercfg-k9qql\""
Jul 13 19:09:50.559856 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.559837    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5d44da\"/\"test-model-5d44da-kserve-self-signed-certs\""
Jul 13 19:09:50.559963 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.559869    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5d44da\"/\"openshift-service-ca.crt\""
Jul 13 19:09:50.559963 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.559942    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5d44da\"/\"kube-root-ca.crt\""
Jul 13 19:09:50.564853 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.564834    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"]
Jul 13 19:09:50.631908 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.631871    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-dshm\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.631908 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.631906    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf926\" (UniqueName: \"kubernetes.io/projected/738ec360-29ac-4e4a-839c-e5743db2498b-kube-api-access-qf926\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.632131 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.631941    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-model-cache\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.632131 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.632054    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-tmp-dir\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.632131 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.632092    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-home\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.632131 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.632127    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738ec360-29ac-4e4a-839c-e5743db2498b-tls-certs\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.733385 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.733350    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-tmp-dir\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.733385 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.733386    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-home\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.733621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.733419    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738ec360-29ac-4e4a-839c-e5743db2498b-tls-certs\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.733621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.733480    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-dshm\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.733621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.733504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf926\" (UniqueName: \"kubernetes.io/projected/738ec360-29ac-4e4a-839c-e5743db2498b-kube-api-access-qf926\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.733621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.733553    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-model-cache\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.733866 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.733842    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-tmp-dir\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.733925 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.733891    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-home\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.734014 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.733995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-model-cache\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.735818 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.735799    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-dshm\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.736231 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.736212    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738ec360-29ac-4e4a-839c-e5743db2498b-tls-certs\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.742418 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.742396    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf926\" (UniqueName: \"kubernetes.io/projected/738ec360-29ac-4e4a-839c-e5743db2498b-kube-api-access-qf926\") pod \"test-model-5d44da-kserve-7698b64bc7-tfjsc\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") " pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:50.867829 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:50.867728    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:09:51.003538 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:51.003514    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"]
Jul 13 19:09:51.005143 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:09:51.005116    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod738ec360_29ac_4e4a_839c_e5743db2498b.slice/crio-abd203b9012213a2aed46eeccf7e6d66e8460af12bd51fd38afcc7a96c04e3c5 WatchSource:0}: Error finding container abd203b9012213a2aed46eeccf7e6d66e8460af12bd51fd38afcc7a96c04e3c5: Status 404 returned error can't find the container with id abd203b9012213a2aed46eeccf7e6d66e8460af12bd51fd38afcc7a96c04e3c5
Jul 13 19:09:51.350583 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:51.350547    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc" event={"ID":"738ec360-29ac-4e4a-839c-e5743db2498b","Type":"ContainerStarted","Data":"8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc"}
Jul 13 19:09:51.350583 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:51.350591    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc" event={"ID":"738ec360-29ac-4e4a-839c-e5743db2498b","Type":"ContainerStarted","Data":"abd203b9012213a2aed46eeccf7e6d66e8460af12bd51fd38afcc7a96c04e3c5"}
Jul 13 19:09:51.388843 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:51.388790    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc" podStartSLOduration=1.388774922 podStartE2EDuration="1.388774922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:09:51.386777213 +0000 UTC m=+2272.518065675" watchObservedRunningTime="2026-07-13 19:09:51.388774922 +0000 UTC m=+2272.520063381"
Jul 13 19:09:55.032175 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:09:55.032141    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:10:00.868274 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:00.868237    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:10:00.868772 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:00.868371    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:10:00.881089 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:00.881064    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:10:01.411850 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:01.411816    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:10:02.068067 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:02.068031    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"]
Jul 13 19:10:02.072543 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:02.072525    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"
Jul 13 19:10:02.075507 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:02.075483    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:10:02.080636 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:02.080613    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"]
Jul 13 19:10:02.230123 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:02.230090    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edaded85-9536-49cc-830b-95bdf909c3c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-p7xwg\" (UID: \"edaded85-9536-49cc-830b-95bdf909c3c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"
Jul 13 19:10:02.331446 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:02.331362    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edaded85-9536-49cc-830b-95bdf909c3c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-p7xwg\" (UID: \"edaded85-9536-49cc-830b-95bdf909c3c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"
Jul 13 19:10:02.332002 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:02.331979    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edaded85-9536-49cc-830b-95bdf909c3c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-p7xwg\" (UID: \"edaded85-9536-49cc-830b-95bdf909c3c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"
Jul 13 19:10:02.383605 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:02.383571    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"
Jul 13 19:10:02.528272 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:02.528250    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"]
Jul 13 19:10:02.530398 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:10:02.530365    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedaded85_9536_49cc_830b_95bdf909c3c7.slice/crio-9e095eece6c7b63866fa64cd3d7bc74e7e629821a353f66edce747f80901e16d WatchSource:0}: Error finding container 9e095eece6c7b63866fa64cd3d7bc74e7e629821a353f66edce747f80901e16d: Status 404 returned error can't find the container with id 9e095eece6c7b63866fa64cd3d7bc74e7e629821a353f66edce747f80901e16d
Jul 13 19:10:03.411616 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:03.411575    2569 generic.go:356] "Generic (PLEG): container finished" podID="edaded85-9536-49cc-830b-95bdf909c3c7" containerID="3a8843d0b7c7c5d6d255da6e2100e35060c36c8e6c70d9b2bd633c380d15828c" exitCode=6
Jul 13 19:10:03.412003 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:03.411673    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg" event={"ID":"edaded85-9536-49cc-830b-95bdf909c3c7","Type":"ContainerDied","Data":"3a8843d0b7c7c5d6d255da6e2100e35060c36c8e6c70d9b2bd633c380d15828c"}
Jul 13 19:10:03.412003 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:03.411733    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg" event={"ID":"edaded85-9536-49cc-830b-95bdf909c3c7","Type":"ContainerStarted","Data":"9e095eece6c7b63866fa64cd3d7bc74e7e629821a353f66edce747f80901e16d"}
Jul 13 19:10:03.412003 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:03.411875    2569 scope.go:117] "RemoveContainer" containerID="3a8843d0b7c7c5d6d255da6e2100e35060c36c8e6c70d9b2bd633c380d15828c"
Jul 13 19:10:04.419105 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:04.419021    2569 generic.go:356] "Generic (PLEG): container finished" podID="edaded85-9536-49cc-830b-95bdf909c3c7" containerID="a28fdd85ddbda4106fc6952e8b764989516993d47a9e0679caacac49cfc78e55" exitCode=6
Jul 13 19:10:04.419499 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:04.419108    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg" event={"ID":"edaded85-9536-49cc-830b-95bdf909c3c7","Type":"ContainerDied","Data":"a28fdd85ddbda4106fc6952e8b764989516993d47a9e0679caacac49cfc78e55"}
Jul 13 19:10:04.419499 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:04.419172    2569 scope.go:117] "RemoveContainer" containerID="3a8843d0b7c7c5d6d255da6e2100e35060c36c8e6c70d9b2bd633c380d15828c"
Jul 13 19:10:04.419499 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:04.419409    2569 scope.go:117] "RemoveContainer" containerID="a28fdd85ddbda4106fc6952e8b764989516993d47a9e0679caacac49cfc78e55"
Jul 13 19:10:04.419665 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:10:04.419636    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-p7xwg_odh-ai-gateway-infra(edaded85-9536-49cc-830b-95bdf909c3c7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg" podUID="edaded85-9536-49cc-830b-95bdf909c3c7"
Jul 13 19:10:05.426057 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:05.426030    2569 scope.go:117] "RemoveContainer" containerID="a28fdd85ddbda4106fc6952e8b764989516993d47a9e0679caacac49cfc78e55"
Jul 13 19:10:05.426433 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:10:05.426207    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-p7xwg_odh-ai-gateway-infra(edaded85-9536-49cc-830b-95bdf909c3c7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg" podUID="edaded85-9536-49cc-830b-95bdf909c3c7"
Jul 13 19:10:07.537398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:07.537365    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:10:18.345393 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:18.345353    2569 scope.go:117] "RemoveContainer" containerID="a28fdd85ddbda4106fc6952e8b764989516993d47a9e0679caacac49cfc78e55"
Jul 13 19:10:19.501830 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:19.501793    2569 generic.go:356] "Generic (PLEG): container finished" podID="edaded85-9536-49cc-830b-95bdf909c3c7" containerID="a3ff5c327e3f4cd6f9a716f076aae218b70008b32b67800162b83dcea9279355" exitCode=6
Jul 13 19:10:19.502287 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:19.501859    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg" event={"ID":"edaded85-9536-49cc-830b-95bdf909c3c7","Type":"ContainerDied","Data":"a3ff5c327e3f4cd6f9a716f076aae218b70008b32b67800162b83dcea9279355"}
Jul 13 19:10:19.502287 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:19.501935    2569 scope.go:117] "RemoveContainer" containerID="a28fdd85ddbda4106fc6952e8b764989516993d47a9e0679caacac49cfc78e55"
Jul 13 19:10:19.502287 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:19.502232    2569 scope.go:117] "RemoveContainer" containerID="a3ff5c327e3f4cd6f9a716f076aae218b70008b32b67800162b83dcea9279355"
Jul 13 19:10:19.502464 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:10:19.502445    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-p7xwg_odh-ai-gateway-infra(edaded85-9536-49cc-830b-95bdf909c3c7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg" podUID="edaded85-9536-49cc-830b-95bdf909c3c7"
Jul 13 19:10:20.531797 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:20.531762    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"]
Jul 13 19:10:20.674795 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:20.674771    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"
Jul 13 19:10:20.785887 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:20.785794    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edaded85-9536-49cc-830b-95bdf909c3c7-maas-api-service-ca\") pod \"edaded85-9536-49cc-830b-95bdf909c3c7\" (UID: \"edaded85-9536-49cc-830b-95bdf909c3c7\") "
Jul 13 19:10:20.786151 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:20.786125    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edaded85-9536-49cc-830b-95bdf909c3c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "edaded85-9536-49cc-830b-95bdf909c3c7" (UID: "edaded85-9536-49cc-830b-95bdf909c3c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:10:20.886938 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:20.886905    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edaded85-9536-49cc-830b-95bdf909c3c7-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:21.516788 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:21.516762    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"
Jul 13 19:10:21.516961 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:21.516787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg" event={"ID":"edaded85-9536-49cc-830b-95bdf909c3c7","Type":"ContainerDied","Data":"9e095eece6c7b63866fa64cd3d7bc74e7e629821a353f66edce747f80901e16d"}
Jul 13 19:10:21.516961 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:21.516846    2569 scope.go:117] "RemoveContainer" containerID="a3ff5c327e3f4cd6f9a716f076aae218b70008b32b67800162b83dcea9279355"
Jul 13 19:10:21.535933 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:21.535908    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"]
Jul 13 19:10:21.542993 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:21.542969    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-p7xwg"]
Jul 13 19:10:23.349947 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:23.349907    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edaded85-9536-49cc-830b-95bdf909c3c7" path="/var/lib/kubelet/pods/edaded85-9536-49cc-830b-95bdf909c3c7/volumes"
Jul 13 19:10:25.539813 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:25.539772    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:10:27.634374 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:27.634344    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"]
Jul 13 19:10:28.686829 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.686792    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7"]
Jul 13 19:10:28.687200 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.687144    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edaded85-9536-49cc-830b-95bdf909c3c7" containerName="revoke-keys"
Jul 13 19:10:28.687200 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.687155    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="edaded85-9536-49cc-830b-95bdf909c3c7" containerName="revoke-keys"
Jul 13 19:10:28.687200 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.687178    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edaded85-9536-49cc-830b-95bdf909c3c7" containerName="revoke-keys"
Jul 13 19:10:28.687200 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.687183    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="edaded85-9536-49cc-830b-95bdf909c3c7" containerName="revoke-keys"
Jul 13 19:10:28.687347 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.687238    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="edaded85-9536-49cc-830b-95bdf909c3c7" containerName="revoke-keys"
Jul 13 19:10:28.687347 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.687248    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="edaded85-9536-49cc-830b-95bdf909c3c7" containerName="revoke-keys"
Jul 13 19:10:28.691618 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.691597    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7"
Jul 13 19:10:28.694255 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.694235    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:10:28.700647 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.700619    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7"]
Jul 13 19:10:28.762842 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.762810    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1aa635-6483-4970-bfd9-bb8eda99ecbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7\" (UID: \"0a1aa635-6483-4970-bfd9-bb8eda99ecbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7"
Jul 13 19:10:28.864117 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.864068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1aa635-6483-4970-bfd9-bb8eda99ecbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7\" (UID: \"0a1aa635-6483-4970-bfd9-bb8eda99ecbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7"
Jul 13 19:10:28.864723 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:28.864703    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1aa635-6483-4970-bfd9-bb8eda99ecbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7\" (UID: \"0a1aa635-6483-4970-bfd9-bb8eda99ecbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7"
Jul 13 19:10:29.003895 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:29.003869    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7"
Jul 13 19:10:29.140119 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:29.140091    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7"]
Jul 13 19:10:29.143347 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:10:29.143320    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a1aa635_6483_4970_bfd9_bb8eda99ecbe.slice/crio-d0f8f3c459a4f66a895070172d66cdc943966560d1e3a13c38bf2e18d5b8537f WatchSource:0}: Error finding container d0f8f3c459a4f66a895070172d66cdc943966560d1e3a13c38bf2e18d5b8537f: Status 404 returned error can't find the container with id d0f8f3c459a4f66a895070172d66cdc943966560d1e3a13c38bf2e18d5b8537f
Jul 13 19:10:29.561134 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:29.561089    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7" event={"ID":"0a1aa635-6483-4970-bfd9-bb8eda99ecbe","Type":"ContainerStarted","Data":"89bf809287445b97d35f13673976de6b06ec9e8d733b0c7a968e0af518274741"}
Jul 13 19:10:29.561314 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:29.561154    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7" event={"ID":"0a1aa635-6483-4970-bfd9-bb8eda99ecbe","Type":"ContainerStarted","Data":"d0f8f3c459a4f66a895070172d66cdc943966560d1e3a13c38bf2e18d5b8537f"}
Jul 13 19:10:29.580811 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:29.580718    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7" podStartSLOduration=1.5807027059999998 podStartE2EDuration="1.580702706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:10:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:10:29.577062622 +0000 UTC m=+2310.708351088" watchObservedRunningTime="2026-07-13 19:10:29.580702706 +0000 UTC m=+2310.711991214"
Jul 13 19:10:30.569442 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:30.569409    2569 generic.go:356] "Generic (PLEG): container finished" podID="0a1aa635-6483-4970-bfd9-bb8eda99ecbe" containerID="89bf809287445b97d35f13673976de6b06ec9e8d733b0c7a968e0af518274741" exitCode=0
Jul 13 19:10:30.569947 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:30.569459    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7" event={"ID":"0a1aa635-6483-4970-bfd9-bb8eda99ecbe","Type":"ContainerDied","Data":"89bf809287445b97d35f13673976de6b06ec9e8d733b0c7a968e0af518274741"}
Jul 13 19:10:31.715021 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:31.714992    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7"
Jul 13 19:10:31.792575 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:31.792544    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1aa635-6483-4970-bfd9-bb8eda99ecbe-maas-api-service-ca\") pod \"0a1aa635-6483-4970-bfd9-bb8eda99ecbe\" (UID: \"0a1aa635-6483-4970-bfd9-bb8eda99ecbe\") "
Jul 13 19:10:31.792904 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:31.792882    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a1aa635-6483-4970-bfd9-bb8eda99ecbe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a1aa635-6483-4970-bfd9-bb8eda99ecbe" (UID: "0a1aa635-6483-4970-bfd9-bb8eda99ecbe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:10:31.894082 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:31.894008    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1aa635-6483-4970-bfd9-bb8eda99ecbe-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:32.580530 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:32.580495    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7" event={"ID":"0a1aa635-6483-4970-bfd9-bb8eda99ecbe","Type":"ContainerDied","Data":"d0f8f3c459a4f66a895070172d66cdc943966560d1e3a13c38bf2e18d5b8537f"}
Jul 13 19:10:32.580530 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:32.580529    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7"
Jul 13 19:10:32.580753 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:32.580565    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0f8f3c459a4f66a895070172d66cdc943966560d1e3a13c38bf2e18d5b8537f"
Jul 13 19:10:37.946074 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:37.946040    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:10:39.162341 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.162306    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"]
Jul 13 19:10:39.162821 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.162521    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g" podUID="eabb1183-0a0e-48e2-bc5c-249e384fc4f8" containerName="maas-api" containerID="cri-o://c79c7b26dd635cc022e278bf99bbe7764d47a46c8902b677bc2d498b98506c4c" gracePeriod=30
Jul 13 19:10:39.253035 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.252997    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g" podUID="eabb1183-0a0e-48e2-bc5c-249e384fc4f8" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.56:8443/health\": dial tcp 10.133.0.56:8443: connect: connection refused"
Jul 13 19:10:39.616350 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.616322    2569 generic.go:356] "Generic (PLEG): container finished" podID="eabb1183-0a0e-48e2-bc5c-249e384fc4f8" containerID="c79c7b26dd635cc022e278bf99bbe7764d47a46c8902b677bc2d498b98506c4c" exitCode=0
Jul 13 19:10:39.616474 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.616395    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g" event={"ID":"eabb1183-0a0e-48e2-bc5c-249e384fc4f8","Type":"ContainerDied","Data":"c79c7b26dd635cc022e278bf99bbe7764d47a46c8902b677bc2d498b98506c4c"}
Jul 13 19:10:39.616474 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.616445    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g" event={"ID":"eabb1183-0a0e-48e2-bc5c-249e384fc4f8","Type":"ContainerDied","Data":"b2499c0804d34eb572450abd16f5f5e76cfa90d8d02899ebaf4993bcea486d42"}
Jul 13 19:10:39.616474 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.616461    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2499c0804d34eb572450abd16f5f5e76cfa90d8d02899ebaf4993bcea486d42"
Jul 13 19:10:39.618430 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.618412    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:10:39.765869 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.765823    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kg6vx\" (UniqueName: \"kubernetes.io/projected/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-kube-api-access-kg6vx\") pod \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") "
Jul 13 19:10:39.765869 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.765876    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-maas-api-tls\") pod \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") "
Jul 13 19:10:39.766122 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.765965    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-openshift-service-ca\") pod \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\" (UID: \"eabb1183-0a0e-48e2-bc5c-249e384fc4f8\") "
Jul 13 19:10:39.766271 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.766246    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eabb1183-0a0e-48e2-bc5c-249e384fc4f8" (UID: "eabb1183-0a0e-48e2-bc5c-249e384fc4f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:10:39.768108 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.768070    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eabb1183-0a0e-48e2-bc5c-249e384fc4f8" (UID: "eabb1183-0a0e-48e2-bc5c-249e384fc4f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:10:39.768233 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.768179    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-kube-api-access-kg6vx" (OuterVolumeSpecName: "kube-api-access-kg6vx") pod "eabb1183-0a0e-48e2-bc5c-249e384fc4f8" (UID: "eabb1183-0a0e-48e2-bc5c-249e384fc4f8"). InnerVolumeSpecName "kube-api-access-kg6vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:10:39.866787 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.866753    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:39.866787 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.866785    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg6vx\" (UniqueName: \"kubernetes.io/projected/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-kube-api-access-kg6vx\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:39.867007 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:39.866799    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eabb1183-0a0e-48e2-bc5c-249e384fc4f8-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:40.621036 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:40.621005    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"
Jul 13 19:10:40.643395 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:40.643364    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"]
Jul 13 19:10:40.649419 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:40.649389    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5d44da-5c748bcd48-ln82g"]
Jul 13 19:10:41.350708 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:41.350675    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eabb1183-0a0e-48e2-bc5c-249e384fc4f8" path="/var/lib/kubelet/pods/eabb1183-0a0e-48e2-bc5c-249e384fc4f8/volumes"
Jul 13 19:10:42.654836 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.654798    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc" podUID="738ec360-29ac-4e4a-839c-e5743db2498b" containerName="main" containerID="cri-o://8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc" gracePeriod=45
Jul 13 19:10:42.904566 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.904542    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:10:42.993742 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.993715    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-dshm\") pod \"738ec360-29ac-4e4a-839c-e5743db2498b\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") "
Jul 13 19:10:42.993902 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.993752    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-tmp-dir\") pod \"738ec360-29ac-4e4a-839c-e5743db2498b\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") "
Jul 13 19:10:42.993902 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.993777    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738ec360-29ac-4e4a-839c-e5743db2498b-tls-certs\") pod \"738ec360-29ac-4e4a-839c-e5743db2498b\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") "
Jul 13 19:10:42.993902 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.993842    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-home\") pod \"738ec360-29ac-4e4a-839c-e5743db2498b\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") "
Jul 13 19:10:42.993902 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.993881    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-model-cache\") pod \"738ec360-29ac-4e4a-839c-e5743db2498b\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") "
Jul 13 19:10:42.994122 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.994025    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qf926\" (UniqueName: \"kubernetes.io/projected/738ec360-29ac-4e4a-839c-e5743db2498b-kube-api-access-qf926\") pod \"738ec360-29ac-4e4a-839c-e5743db2498b\" (UID: \"738ec360-29ac-4e4a-839c-e5743db2498b\") "
Jul 13 19:10:42.994122 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.994047    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "738ec360-29ac-4e4a-839c-e5743db2498b" (UID: "738ec360-29ac-4e4a-839c-e5743db2498b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:10:42.994227 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.994128    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-model-cache" (OuterVolumeSpecName: "model-cache") pod "738ec360-29ac-4e4a-839c-e5743db2498b" (UID: "738ec360-29ac-4e4a-839c-e5743db2498b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:10:42.994227 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.994148    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-home" (OuterVolumeSpecName: "home") pod "738ec360-29ac-4e4a-839c-e5743db2498b" (UID: "738ec360-29ac-4e4a-839c-e5743db2498b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:10:42.994353 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.994337    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-home\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:42.994411 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.994359    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-model-cache\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:42.994411 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.994375    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-tmp-dir\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:42.996041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.996020    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-dshm" (OuterVolumeSpecName: "dshm") pod "738ec360-29ac-4e4a-839c-e5743db2498b" (UID: "738ec360-29ac-4e4a-839c-e5743db2498b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:10:42.996166 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.996101    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/738ec360-29ac-4e4a-839c-e5743db2498b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "738ec360-29ac-4e4a-839c-e5743db2498b" (UID: "738ec360-29ac-4e4a-839c-e5743db2498b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:10:42.996166 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:42.996101    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/738ec360-29ac-4e4a-839c-e5743db2498b-kube-api-access-qf926" (OuterVolumeSpecName: "kube-api-access-qf926") pod "738ec360-29ac-4e4a-839c-e5743db2498b" (UID: "738ec360-29ac-4e4a-839c-e5743db2498b"). InnerVolumeSpecName "kube-api-access-qf926". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:10:43.095339 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.095305    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/738ec360-29ac-4e4a-839c-e5743db2498b-dshm\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:43.095339 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.095330    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738ec360-29ac-4e4a-839c-e5743db2498b-tls-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:43.095339 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.095342    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf926\" (UniqueName: \"kubernetes.io/projected/738ec360-29ac-4e4a-839c-e5743db2498b-kube-api-access-qf926\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:43.641779 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.641685    2569 generic.go:356] "Generic (PLEG): container finished" podID="738ec360-29ac-4e4a-839c-e5743db2498b" containerID="8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc" exitCode=0
Jul 13 19:10:43.641926 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.641806    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"
Jul 13 19:10:43.641926 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.641822    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc" event={"ID":"738ec360-29ac-4e4a-839c-e5743db2498b","Type":"ContainerDied","Data":"8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc"}
Jul 13 19:10:43.641926 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.641866    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc" event={"ID":"738ec360-29ac-4e4a-839c-e5743db2498b","Type":"ContainerDied","Data":"abd203b9012213a2aed46eeccf7e6d66e8460af12bd51fd38afcc7a96c04e3c5"}
Jul 13 19:10:43.641926 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.641884    2569 scope.go:117] "RemoveContainer" containerID="8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc"
Jul 13 19:10:43.650852 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.650831    2569 scope.go:117] "RemoveContainer" containerID="8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc"
Jul 13 19:10:43.651118 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:10:43.651099    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc\": container with ID starting with 8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc not found: ID does not exist" containerID="8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc"
Jul 13 19:10:43.651193 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.651127    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc"} err="failed to get container status \"8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc\": rpc error: code = NotFound desc = could not find container \"8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc\": container with ID starting with 8ea06e30e0be11fa69e5e09e4918c4dede1f4e8eb59beed9cbbe9b84f81898cc not found: ID does not exist"
Jul 13 19:10:43.662198 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.662177    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"]
Jul 13 19:10:43.664731 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:43.664711    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-5d44da/test-model-5d44da-kserve-7698b64bc7-tfjsc"]
Jul 13 19:10:45.350589 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:45.350557    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="738ec360-29ac-4e4a-839c-e5743db2498b" path="/var/lib/kubelet/pods/738ec360-29ac-4e4a-839c-e5743db2498b/volumes"
Jul 13 19:10:47.431490 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:47.431461    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:10:52.099160 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099076    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"]
Jul 13 19:10:52.099600 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099553    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edaded85-9536-49cc-830b-95bdf909c3c7" containerName="revoke-keys"
Jul 13 19:10:52.099600 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099571    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="edaded85-9536-49cc-830b-95bdf909c3c7" containerName="revoke-keys"
Jul 13 19:10:52.099600 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099583    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a1aa635-6483-4970-bfd9-bb8eda99ecbe" containerName="revoke-keys"
Jul 13 19:10:52.099600 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099591    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a1aa635-6483-4970-bfd9-bb8eda99ecbe" containerName="revoke-keys"
Jul 13 19:10:52.099872 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099606    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="738ec360-29ac-4e4a-839c-e5743db2498b" containerName="main"
Jul 13 19:10:52.099872 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099614    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="738ec360-29ac-4e4a-839c-e5743db2498b" containerName="main"
Jul 13 19:10:52.099872 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099641    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eabb1183-0a0e-48e2-bc5c-249e384fc4f8" containerName="maas-api"
Jul 13 19:10:52.099872 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099670    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eabb1183-0a0e-48e2-bc5c-249e384fc4f8" containerName="maas-api"
Jul 13 19:10:52.099872 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099765    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a1aa635-6483-4970-bfd9-bb8eda99ecbe" containerName="revoke-keys"
Jul 13 19:10:52.099872 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099778    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eabb1183-0a0e-48e2-bc5c-249e384fc4f8" containerName="maas-api"
Jul 13 19:10:52.099872 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099788    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="edaded85-9536-49cc-830b-95bdf909c3c7" containerName="revoke-keys"
Jul 13 19:10:52.099872 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.099798    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="738ec360-29ac-4e4a-839c-e5743db2498b" containerName="main"
Jul 13 19:10:52.104145 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.104123    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"
Jul 13 19:10:52.106736 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.106712    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:10:52.109668 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.109627    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"]
Jul 13 19:10:52.273133 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.273094    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-gkw84\" (UID: \"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"
Jul 13 19:10:52.373612 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.373531    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-gkw84\" (UID: \"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"
Jul 13 19:10:52.374168 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.374148    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-gkw84\" (UID: \"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"
Jul 13 19:10:52.414983 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.414956    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"
Jul 13 19:10:52.552764 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.552738    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"]
Jul 13 19:10:52.554764 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:10:52.554735    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ef8dce_e6dc_4802_ae5e_6b7e179ccc8e.slice/crio-55fa87f99e18903de11d9d6771b511497ebb103ed88b6137c21759aa742f45dc WatchSource:0}: Error finding container 55fa87f99e18903de11d9d6771b511497ebb103ed88b6137c21759aa742f45dc: Status 404 returned error can't find the container with id 55fa87f99e18903de11d9d6771b511497ebb103ed88b6137c21759aa742f45dc
Jul 13 19:10:52.690809 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:52.690736    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84" event={"ID":"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e","Type":"ContainerStarted","Data":"55fa87f99e18903de11d9d6771b511497ebb103ed88b6137c21759aa742f45dc"}
Jul 13 19:10:53.697023 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:53.696988    2569 generic.go:356] "Generic (PLEG): container finished" podID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerID="19ed824cc81a72162d3d1c2a3c1a25bfb4b9e4ad034ba0808041eeb8470c5d94" exitCode=6
Jul 13 19:10:53.697023 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:53.697027    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84" event={"ID":"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e","Type":"ContainerDied","Data":"19ed824cc81a72162d3d1c2a3c1a25bfb4b9e4ad034ba0808041eeb8470c5d94"}
Jul 13 19:10:53.697427 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:53.697247    2569 scope.go:117] "RemoveContainer" containerID="19ed824cc81a72162d3d1c2a3c1a25bfb4b9e4ad034ba0808041eeb8470c5d94"
Jul 13 19:10:54.704351 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:54.704261    2569 generic.go:356] "Generic (PLEG): container finished" podID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerID="fbd2c91a1667b5998e12781d18df54d3b0ede5202bba1a4eb70b595e08b9cb72" exitCode=6
Jul 13 19:10:54.704770 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:54.704346    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84" event={"ID":"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e","Type":"ContainerDied","Data":"fbd2c91a1667b5998e12781d18df54d3b0ede5202bba1a4eb70b595e08b9cb72"}
Jul 13 19:10:54.704770 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:54.704404    2569 scope.go:117] "RemoveContainer" containerID="19ed824cc81a72162d3d1c2a3c1a25bfb4b9e4ad034ba0808041eeb8470c5d94"
Jul 13 19:10:54.704770 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:54.704547    2569 scope.go:117] "RemoveContainer" containerID="fbd2c91a1667b5998e12781d18df54d3b0ede5202bba1a4eb70b595e08b9cb72"
Jul 13 19:10:54.704882 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:10:54.704791    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-gkw84_odh-ai-gateway-infra(f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e"
Jul 13 19:10:54.858043 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:54.858010    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c"]
Jul 13 19:10:54.862547 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:54.862531    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c"
Jul 13 19:10:54.870959 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:54.870937    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c"]
Jul 13 19:10:54.997405 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:54.997371    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c\" (UID: \"02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c"
Jul 13 19:10:55.098684 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:55.098634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c\" (UID: \"02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c"
Jul 13 19:10:55.099247 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:55.099227    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c\" (UID: \"02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c"
Jul 13 19:10:55.174538 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:55.174510    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c"
Jul 13 19:10:55.338126 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:55.338096    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c"]
Jul 13 19:10:55.344105 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:10:55.344077    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d3e594_bc5a_4f4e_b1d1_25d0d045bf3a.slice/crio-f21704cb981209484586d0ccebad822b94358059bd747e076f1c212e60262c74 WatchSource:0}: Error finding container f21704cb981209484586d0ccebad822b94358059bd747e076f1c212e60262c74: Status 404 returned error can't find the container with id f21704cb981209484586d0ccebad822b94358059bd747e076f1c212e60262c74
Jul 13 19:10:55.711300 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:55.711260    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c" event={"ID":"02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a","Type":"ContainerStarted","Data":"ed295abb467e3ac771e413f316530da9394f40d695a2985bb4da677a7893ac66"}
Jul 13 19:10:55.711705 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:55.711326    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c" event={"ID":"02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a","Type":"ContainerStarted","Data":"f21704cb981209484586d0ccebad822b94358059bd747e076f1c212e60262c74"}
Jul 13 19:10:55.713049 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:55.713028    2569 scope.go:117] "RemoveContainer" containerID="fbd2c91a1667b5998e12781d18df54d3b0ede5202bba1a4eb70b595e08b9cb72"
Jul 13 19:10:55.713205 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:10:55.713188    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-gkw84_odh-ai-gateway-infra(f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e"
Jul 13 19:10:55.728226 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:55.728153    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c" podStartSLOduration=1.728133155 podStartE2EDuration="1.728133155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:10:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:10:55.726634481 +0000 UTC m=+2336.857922940" watchObservedRunningTime="2026-07-13 19:10:55.728133155 +0000 UTC m=+2336.859421616"
Jul 13 19:10:56.721378 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:56.721342    2569 generic.go:356] "Generic (PLEG): container finished" podID="02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a" containerID="ed295abb467e3ac771e413f316530da9394f40d695a2985bb4da677a7893ac66" exitCode=0
Jul 13 19:10:56.721768 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:56.721426    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c" event={"ID":"02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a","Type":"ContainerDied","Data":"ed295abb467e3ac771e413f316530da9394f40d695a2985bb4da677a7893ac66"}
Jul 13 19:10:57.859609 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:57.859585    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c"
Jul 13 19:10:57.925617 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:57.925592    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a-maas-api-service-ca\") pod \"02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a\" (UID: \"02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a\") "
Jul 13 19:10:57.925894 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:57.925872    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a" (UID: "02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:10:58.027107 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:58.027022    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:10:58.733793 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:58.733767    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c"
Jul 13 19:10:58.733965 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:58.733760    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c" event={"ID":"02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a","Type":"ContainerDied","Data":"f21704cb981209484586d0ccebad822b94358059bd747e076f1c212e60262c74"}
Jul 13 19:10:58.733965 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:10:58.733888    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f21704cb981209484586d0ccebad822b94358059bd747e076f1c212e60262c74"
Jul 13 19:11:05.359491 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.359457    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"]
Jul 13 19:11:05.359948 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.359640    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx" podUID="4d4cbb70-75bb-444d-895b-34d99d6e8f17" containerName="maas-api" containerID="cri-o://3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa" gracePeriod=30
Jul 13 19:11:05.624768 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.624742    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:11:05.694779 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.694747    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4cbb70-75bb-444d-895b-34d99d6e8f17-maas-api-tls\") pod \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") "
Jul 13 19:11:05.694945 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.694816    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r8bj\" (UniqueName: \"kubernetes.io/projected/4d4cbb70-75bb-444d-895b-34d99d6e8f17-kube-api-access-8r8bj\") pod \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") "
Jul 13 19:11:05.694945 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.694867    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4cbb70-75bb-444d-895b-34d99d6e8f17-openshift-service-ca\") pod \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\" (UID: \"4d4cbb70-75bb-444d-895b-34d99d6e8f17\") "
Jul 13 19:11:05.695263 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.695241    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d4cbb70-75bb-444d-895b-34d99d6e8f17-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d4cbb70-75bb-444d-895b-34d99d6e8f17" (UID: "4d4cbb70-75bb-444d-895b-34d99d6e8f17"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:11:05.697063 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.697039    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d4cbb70-75bb-444d-895b-34d99d6e8f17-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d4cbb70-75bb-444d-895b-34d99d6e8f17" (UID: "4d4cbb70-75bb-444d-895b-34d99d6e8f17"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:11:05.697150 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.697081    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d4cbb70-75bb-444d-895b-34d99d6e8f17-kube-api-access-8r8bj" (OuterVolumeSpecName: "kube-api-access-8r8bj") pod "4d4cbb70-75bb-444d-895b-34d99d6e8f17" (UID: "4d4cbb70-75bb-444d-895b-34d99d6e8f17"). InnerVolumeSpecName "kube-api-access-8r8bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:11:05.778590 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.778555    2569 generic.go:356] "Generic (PLEG): container finished" podID="4d4cbb70-75bb-444d-895b-34d99d6e8f17" containerID="3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa" exitCode=0
Jul 13 19:11:05.778760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.778602    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"
Jul 13 19:11:05.778760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.778640    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx" event={"ID":"4d4cbb70-75bb-444d-895b-34d99d6e8f17","Type":"ContainerDied","Data":"3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa"}
Jul 13 19:11:05.778760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.778723    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx" event={"ID":"4d4cbb70-75bb-444d-895b-34d99d6e8f17","Type":"ContainerDied","Data":"33108f278fbcffeecfa40a46d9479b75eb3a45d213fc3529cd5fd10fd6903420"}
Jul 13 19:11:05.778760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.778755    2569 scope.go:117] "RemoveContainer" containerID="3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa"
Jul 13 19:11:05.788274 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.788253    2569 scope.go:117] "RemoveContainer" containerID="3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa"
Jul 13 19:11:05.788541 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:11:05.788522    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa\": container with ID starting with 3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa not found: ID does not exist" containerID="3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa"
Jul 13 19:11:05.788590 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.788550    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa"} err="failed to get container status \"3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa\": rpc error: code = NotFound desc = could not find container \"3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa\": container with ID starting with 3f74f11a302d9361ceb1bef9ff33db71df9dda83ae3dc8f3893b9df110bac7aa not found: ID does not exist"
Jul 13 19:11:05.796303 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.796281    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r8bj\" (UniqueName: \"kubernetes.io/projected/4d4cbb70-75bb-444d-895b-34d99d6e8f17-kube-api-access-8r8bj\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:11:05.796361 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.796301    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4cbb70-75bb-444d-895b-34d99d6e8f17-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:11:05.796361 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.796312    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d4cbb70-75bb-444d-895b-34d99d6e8f17-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:11:05.802477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.802452    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"]
Jul 13 19:11:05.810169 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:05.810151    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a46d9-5d7ffd9c4c-m69kx"]
Jul 13 19:11:07.354458 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:07.354424    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d4cbb70-75bb-444d-895b-34d99d6e8f17" path="/var/lib/kubelet/pods/4d4cbb70-75bb-444d-895b-34d99d6e8f17/volumes"
Jul 13 19:11:08.344950 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:08.344918    2569 scope.go:117] "RemoveContainer" containerID="fbd2c91a1667b5998e12781d18df54d3b0ede5202bba1a4eb70b595e08b9cb72"
Jul 13 19:11:08.346411 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:08.346337    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:11:08.797084 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:08.797047    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84" event={"ID":"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e","Type":"ContainerStarted","Data":"7477e23246ab04c34b0d803f755473a7d43a310ac404badc46ccc98cde03ee70"}
Jul 13 19:11:08.816052 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:08.815998    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84" podStartSLOduration=16.815982013 podStartE2EDuration="16.815982013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:11:08.814254878 +0000 UTC m=+2349.945543342" watchObservedRunningTime="2026-07-13 19:11:08.815982013 +0000 UTC m=+2349.947270496"
Jul 13 19:11:09.804209 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:09.804175    2569 generic.go:356] "Generic (PLEG): container finished" podID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerID="7477e23246ab04c34b0d803f755473a7d43a310ac404badc46ccc98cde03ee70" exitCode=6
Jul 13 19:11:09.804704 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:09.804252    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84" event={"ID":"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e","Type":"ContainerDied","Data":"7477e23246ab04c34b0d803f755473a7d43a310ac404badc46ccc98cde03ee70"}
Jul 13 19:11:09.804704 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:09.804313    2569 scope.go:117] "RemoveContainer" containerID="fbd2c91a1667b5998e12781d18df54d3b0ede5202bba1a4eb70b595e08b9cb72"
Jul 13 19:11:09.804704 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:09.804571    2569 scope.go:117] "RemoveContainer" containerID="7477e23246ab04c34b0d803f755473a7d43a310ac404badc46ccc98cde03ee70"
Jul 13 19:11:09.804911 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:11:09.804788    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-gkw84_odh-ai-gateway-infra(f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e"
Jul 13 19:11:09.834154 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:09.834127    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"]
Jul 13 19:11:10.946900 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:10.946874    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"
Jul 13 19:11:11.040349 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:11.040320    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e-maas-api-service-ca\") pod \"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e\" (UID: \"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e\") "
Jul 13 19:11:11.040707 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:11.040686    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" (UID: "f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:11:11.141492 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:11.141405    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:11:11.817944 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:11.817912    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"
Jul 13 19:11:11.818144 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:11.817948    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84" event={"ID":"f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e","Type":"ContainerDied","Data":"55fa87f99e18903de11d9d6771b511497ebb103ed88b6137c21759aa742f45dc"}
Jul 13 19:11:11.818144 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:11.818013    2569 scope.go:117] "RemoveContainer" containerID="7477e23246ab04c34b0d803f755473a7d43a310ac404badc46ccc98cde03ee70"
Jul 13 19:11:11.836452 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:11.836426    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"]
Jul 13 19:11:11.838574 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:11.838557    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-gkw84"]
Jul 13 19:11:13.350192 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:13.350157    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" path="/var/lib/kubelet/pods/f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e/volumes"
Jul 13 19:11:21.619377 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619336    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"]
Jul 13 19:11:21.619760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619708    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerName="revoke-keys"
Jul 13 19:11:21.619760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619719    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerName="revoke-keys"
Jul 13 19:11:21.619760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619738    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a" containerName="revoke-keys"
Jul 13 19:11:21.619760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619747    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a" containerName="revoke-keys"
Jul 13 19:11:21.619760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619755    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerName="revoke-keys"
Jul 13 19:11:21.619760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619760    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerName="revoke-keys"
Jul 13 19:11:21.619949 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619768    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d4cbb70-75bb-444d-895b-34d99d6e8f17" containerName="maas-api"
Jul 13 19:11:21.619949 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619773    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d4cbb70-75bb-444d-895b-34d99d6e8f17" containerName="maas-api"
Jul 13 19:11:21.619949 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619778    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerName="revoke-keys"
Jul 13 19:11:21.619949 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619782    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerName="revoke-keys"
Jul 13 19:11:21.619949 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619858    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerName="revoke-keys"
Jul 13 19:11:21.619949 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619866    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerName="revoke-keys"
Jul 13 19:11:21.619949 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619873    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5ef8dce-e6dc-4802-ae5e-6b7e179ccc8e" containerName="revoke-keys"
Jul 13 19:11:21.619949 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619879    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a" containerName="revoke-keys"
Jul 13 19:11:21.619949 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.619887    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d4cbb70-75bb-444d-895b-34d99d6e8f17" containerName="maas-api"
Jul 13 19:11:21.624206 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.624184    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.638042 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.638022    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e196f1-openshift-default-dockercfg-f2qfg\""
Jul 13 19:11:21.673101 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.673072    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"]
Jul 13 19:11:21.734887 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.734853    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-socket\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.734887 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.734891    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vlv9\" (UniqueName: \"kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-kube-api-access-7vlv9\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.735059 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.734911    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-certs\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.735059 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.734949    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-podinfo\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.735059 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.734998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-credential-socket\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.735059 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.735014    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-envoy\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.735059 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.735038    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-token\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.735236 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.735093    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-data\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.735236 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.735130    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6410559f-81bd-48ca-911d-da11c4d52a3f-istiod-ca-cert\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.835866 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.835827    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-data\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.835871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6410559f-81bd-48ca-911d-da11c4d52a3f-istiod-ca-cert\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.835919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-socket\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.835937    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vlv9\" (UniqueName: \"kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-kube-api-access-7vlv9\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.835956    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-certs\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.835970    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-podinfo\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836041 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.835997    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-credential-socket\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.836142    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-envoy\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.836185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-token\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.836279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-data\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.836340    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-socket\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836585 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.836388    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-certs\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836585 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.836448    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-credential-socket\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.836991 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.836968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6410559f-81bd-48ca-911d-da11c4d52a3f-istiod-ca-cert\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.838595 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.838579    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-envoy\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.838777 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.838761    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-podinfo\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.846061 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.846041    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vlv9\" (UniqueName: \"kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-kube-api-access-7vlv9\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.846795 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.846779    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-token\") pod \"e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") " pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:21.934136 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:21.934056    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:22.086142 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:22.086117    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"]
Jul 13 19:11:22.088098 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:11:22.088065    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6410559f_81bd_48ca_911d_da11c4d52a3f.slice/crio-4fe4e28aac634d443ee2e79bd1f7b8bfc7abbc8b4710c19bf9e37966a244982d WatchSource:0}: Error finding container 4fe4e28aac634d443ee2e79bd1f7b8bfc7abbc8b4710c19bf9e37966a244982d: Status 404 returned error can't find the container with id 4fe4e28aac634d443ee2e79bd1f7b8bfc7abbc8b4710c19bf9e37966a244982d
Jul 13 19:11:22.877237 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:22.877195    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" event={"ID":"6410559f-81bd-48ca-911d-da11c4d52a3f","Type":"ContainerStarted","Data":"4fe4e28aac634d443ee2e79bd1f7b8bfc7abbc8b4710c19bf9e37966a244982d"}
Jul 13 19:11:24.666759 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:24.666721    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:11:24.667004 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:24.666798    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:11:24.667004 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:24.666825    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:11:24.891052 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:24.891000    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" event={"ID":"6410559f-81bd-48ca-911d-da11c4d52a3f","Type":"ContainerStarted","Data":"8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab"}
Jul 13 19:11:24.916196 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:24.916085    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" podStartSLOduration=1.3397342 podStartE2EDuration="3.916058169s" totalImagesPullingTime="2.576323969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:11:24.914025801 +0000 UTC m=+2366.045314259" watchObservedRunningTime="2026-07-13 19:11:24.916058169 +0000 UTC m=+2366.047346630"
Jul 13 19:11:24.934960 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:24.934929    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:25.939834 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:25.939805    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:26.901737 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:26.901701    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:26.902597 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:26.902578    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:27.039790 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.039756    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"]
Jul 13 19:11:27.044019 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.043997    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.046625 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.046591    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 13 19:11:27.046625 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.046610    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:11:27.046625 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.046610    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xfcfp\""
Jul 13 19:11:27.046902 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.046882    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-e196f1-serving-cert\""
Jul 13 19:11:27.058291 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.058265    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"]
Jul 13 19:11:27.186591 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.186494    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05c6fab-9561-484f-a452-eaf0d1d6ea48-maas-api-tls\") pod \"maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.186591 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.186537    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b05c6fab-9561-484f-a452-eaf0d1d6ea48-openshift-service-ca\") pod \"maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.186817 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.186607    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24nzc\" (UniqueName: \"kubernetes.io/projected/b05c6fab-9561-484f-a452-eaf0d1d6ea48-kube-api-access-24nzc\") pod \"maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.287562 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.287530    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05c6fab-9561-484f-a452-eaf0d1d6ea48-maas-api-tls\") pod \"maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.287562 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.287565    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b05c6fab-9561-484f-a452-eaf0d1d6ea48-openshift-service-ca\") pod \"maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.287854 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.287594    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24nzc\" (UniqueName: \"kubernetes.io/projected/b05c6fab-9561-484f-a452-eaf0d1d6ea48-kube-api-access-24nzc\") pod \"maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.287854 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:11:27.287701    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-serving-cert: secret "maas-api-e2e-api-a-e196f1-serving-cert" not found
Jul 13 19:11:27.287854 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:11:27.287769    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05c6fab-9561-484f-a452-eaf0d1d6ea48-maas-api-tls podName:b05c6fab-9561-484f-a452-eaf0d1d6ea48 nodeName:}" failed. No retries permitted until 2026-07-13 19:11:27.787749663 +0000 UTC m=+2368.919038102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b05c6fab-9561-484f-a452-eaf0d1d6ea48-maas-api-tls") pod "maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x" (UID: "b05c6fab-9561-484f-a452-eaf0d1d6ea48") : secret "maas-api-e2e-api-a-e196f1-serving-cert" not found
Jul 13 19:11:27.288308 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.288283    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b05c6fab-9561-484f-a452-eaf0d1d6ea48-openshift-service-ca\") pod \"maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.297295 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.297270    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24nzc\" (UniqueName: \"kubernetes.io/projected/b05c6fab-9561-484f-a452-eaf0d1d6ea48-kube-api-access-24nzc\") pod \"maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.792438 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.792399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05c6fab-9561-484f-a452-eaf0d1d6ea48-maas-api-tls\") pod \"maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.795107 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.795076    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05c6fab-9561-484f-a452-eaf0d1d6ea48-maas-api-tls\") pod \"maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:27.954338 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:27.954300    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:28.096775 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:28.096701    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"]
Jul 13 19:11:28.100065 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:11:28.100026    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05c6fab_9561_484f_a452_eaf0d1d6ea48.slice/crio-45044b69c15b4b4a5a9b7a35120ab30d32ee4499d09bf262dbd97e2e8af35a5a WatchSource:0}: Error finding container 45044b69c15b4b4a5a9b7a35120ab30d32ee4499d09bf262dbd97e2e8af35a5a: Status 404 returned error can't find the container with id 45044b69c15b4b4a5a9b7a35120ab30d32ee4499d09bf262dbd97e2e8af35a5a
Jul 13 19:11:28.914575 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:28.914475    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x" event={"ID":"b05c6fab-9561-484f-a452-eaf0d1d6ea48","Type":"ContainerStarted","Data":"2d682289b4a3e87a86cbf79f6a41646ca34e2c5fb532f78b233dfb1ecf88b080"}
Jul 13 19:11:28.914575 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:28.914545    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x" event={"ID":"b05c6fab-9561-484f-a452-eaf0d1d6ea48","Type":"ContainerStarted","Data":"45044b69c15b4b4a5a9b7a35120ab30d32ee4499d09bf262dbd97e2e8af35a5a"}
Jul 13 19:11:28.942758 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:28.942709    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x" podStartSLOduration=1.5005033060000001 podStartE2EDuration="1.94269421s" totalImagesPullingTime="442.190904ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:11:28.939865515 +0000 UTC m=+2370.071153975" watchObservedRunningTime="2026-07-13 19:11:28.94269421 +0000 UTC m=+2370.073982669"
Jul 13 19:11:29.919823 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:29.919787    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:30.845634 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:30.845601    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:11:35.417000 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.416968    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"]
Jul 13 19:11:35.421130 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.421098    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:35.425673 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.425146    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-5ffdec-serving-cert\""
Jul 13 19:11:35.440180 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.440152    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"]
Jul 13 19:11:35.560709 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.560666    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a672e79c-fb0c-4155-9f22-da5af2b84efe-openshift-service-ca\") pod \"maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:35.560863 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.560752    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x97vc\" (UniqueName: \"kubernetes.io/projected/a672e79c-fb0c-4155-9f22-da5af2b84efe-kube-api-access-x97vc\") pod \"maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:35.560863 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.560776    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a672e79c-fb0c-4155-9f22-da5af2b84efe-maas-api-tls\") pod \"maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:35.662037 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.662002    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x97vc\" (UniqueName: \"kubernetes.io/projected/a672e79c-fb0c-4155-9f22-da5af2b84efe-kube-api-access-x97vc\") pod \"maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:35.662037 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.662041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a672e79c-fb0c-4155-9f22-da5af2b84efe-maas-api-tls\") pod \"maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:35.662278 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.662113    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a672e79c-fb0c-4155-9f22-da5af2b84efe-openshift-service-ca\") pod \"maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:35.662278 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:11:35.662226    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-serving-cert: secret "maas-api-e2e-api-b-5ffdec-serving-cert" not found
Jul 13 19:11:35.662365 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:11:35.662303    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a672e79c-fb0c-4155-9f22-da5af2b84efe-maas-api-tls podName:a672e79c-fb0c-4155-9f22-da5af2b84efe nodeName:}" failed. No retries permitted until 2026-07-13 19:11:36.162284233 +0000 UTC m=+2377.293572672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a672e79c-fb0c-4155-9f22-da5af2b84efe-maas-api-tls") pod "maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2" (UID: "a672e79c-fb0c-4155-9f22-da5af2b84efe") : secret "maas-api-e2e-api-b-5ffdec-serving-cert" not found
Jul 13 19:11:35.663618 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.663583    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a672e79c-fb0c-4155-9f22-da5af2b84efe-openshift-service-ca\") pod \"maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:35.674874 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.674810    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x97vc\" (UniqueName: \"kubernetes.io/projected/a672e79c-fb0c-4155-9f22-da5af2b84efe-kube-api-access-x97vc\") pod \"maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:35.930861 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:35.930782    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:36.166871 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:36.166836    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a672e79c-fb0c-4155-9f22-da5af2b84efe-maas-api-tls\") pod \"maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:36.171803 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:36.171775    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a672e79c-fb0c-4155-9f22-da5af2b84efe-maas-api-tls\") pod \"maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:36.336678 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:36.336636    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:36.514136 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:36.514108    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"]
Jul 13 19:11:36.516617 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:11:36.516586    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda672e79c_fb0c_4155_9f22_da5af2b84efe.slice/crio-8e783dc0cbd3290bf4f3e40fef9fca0d97d1cd0d10719399360ec980ec5a5c04 WatchSource:0}: Error finding container 8e783dc0cbd3290bf4f3e40fef9fca0d97d1cd0d10719399360ec980ec5a5c04: Status 404 returned error can't find the container with id 8e783dc0cbd3290bf4f3e40fef9fca0d97d1cd0d10719399360ec980ec5a5c04
Jul 13 19:11:36.957728 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:36.957692    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2" event={"ID":"a672e79c-fb0c-4155-9f22-da5af2b84efe","Type":"ContainerStarted","Data":"8e783dc0cbd3290bf4f3e40fef9fca0d97d1cd0d10719399360ec980ec5a5c04"}
Jul 13 19:11:37.965311 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:37.965276    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2" event={"ID":"a672e79c-fb0c-4155-9f22-da5af2b84efe","Type":"ContainerStarted","Data":"4d65db44428bfcc2649749ec344a56a36c580fde6fb1c8ef59374cc11252da39"}
Jul 13 19:11:37.965744 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:37.965336    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:37.990999 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:37.990940    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2" podStartSLOduration=2.417794679 podStartE2EDuration="2.990926098s" totalImagesPullingTime="573.131419ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:11:37.990548918 +0000 UTC m=+2379.121837377" watchObservedRunningTime="2026-07-13 19:11:37.990926098 +0000 UTC m=+2379.122214558"
Jul 13 19:11:38.971910 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:38.971826    2569 generic.go:356] "Generic (PLEG): container finished" podID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerID="8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab" exitCode=0
Jul 13 19:11:38.971910 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:38.971896    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" event={"ID":"6410559f-81bd-48ca-911d-da11c4d52a3f","Type":"ContainerDied","Data":"8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab"}
Jul 13 19:11:38.972393 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:38.972375    2569 scope.go:117] "RemoveContainer" containerID="8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab"
Jul 13 19:11:38.973360 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:38.973328    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:11:38.973416 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:38.973402    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:11:38.973456 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:38.973436    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:11:39.979690 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:39.979633    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" event={"ID":"6410559f-81bd-48ca-911d-da11c4d52a3f","Type":"ContainerStarted","Data":"26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783"}
Jul 13 19:11:40.934345 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:40.934313    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:40.935304 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:40.935281    2569 patch_prober.go:28] interesting pod/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 13 19:11:40.935411 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:40.935328    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 13 19:11:41.902438 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:41.902392    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:41.934799 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:41.934761    2569 patch_prober.go:28] interesting pod/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 13 19:11:41.934970 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:41.934822    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 13 19:11:42.150456 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.150425    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"]
Jul 13 19:11:42.157313 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.157250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"
Jul 13 19:11:42.159888 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.159864    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:11:42.162315 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.162289    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"]
Jul 13 19:11:42.330610 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.330569    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-jgdsk\" (UID: \"20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"
Jul 13 19:11:42.432092 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.431986    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-jgdsk\" (UID: \"20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"
Jul 13 19:11:42.432610 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.432591    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-jgdsk\" (UID: \"20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"
Jul 13 19:11:42.468975 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.468944    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"
Jul 13 19:11:42.670579 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.670545    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"]
Jul 13 19:11:42.673228 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:11:42.673195    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20bdaf22_4e8d_4be1_bb7c_53e08d7b25f6.slice/crio-46b6c646b37320e654d048cd8363fd917ce2fec9d01fdc1dd50f40abc3ad3ef4 WatchSource:0}: Error finding container 46b6c646b37320e654d048cd8363fd917ce2fec9d01fdc1dd50f40abc3ad3ef4: Status 404 returned error can't find the container with id 46b6c646b37320e654d048cd8363fd917ce2fec9d01fdc1dd50f40abc3ad3ef4
Jul 13 19:11:42.935011 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.934975    2569 patch_prober.go:28] interesting pod/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 13 19:11:42.935360 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.935032    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 13 19:11:42.996600 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:42.996567    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk" event={"ID":"20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6","Type":"ContainerStarted","Data":"46b6c646b37320e654d048cd8363fd917ce2fec9d01fdc1dd50f40abc3ad3ef4"}
Jul 13 19:11:43.934495 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:43.934403    2569 patch_prober.go:28] interesting pod/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 13 19:11:43.934674 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:43.934490    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 13 19:11:43.977443 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:43.977416    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:11:44.007783 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:44.007744    2569 generic.go:356] "Generic (PLEG): container finished" podID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerID="74d9ad2057e100f5208e1b5075c9f1daefa590caecfd5713350765f6a5e2904b" exitCode=6
Jul 13 19:11:44.007979 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:44.007812    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk" event={"ID":"20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6","Type":"ContainerDied","Data":"74d9ad2057e100f5208e1b5075c9f1daefa590caecfd5713350765f6a5e2904b"}
Jul 13 19:11:44.008193 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:44.008170    2569 scope.go:117] "RemoveContainer" containerID="74d9ad2057e100f5208e1b5075c9f1daefa590caecfd5713350765f6a5e2904b"
Jul 13 19:11:44.939268 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:44.939185    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:44.940196 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:44.940177    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:11:45.017553 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:45.017518    2569 generic.go:356] "Generic (PLEG): container finished" podID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerID="c51768794361b00da3dbc50791710842b2c2f3648eb651176b77d07bff023358" exitCode=6
Jul 13 19:11:45.018042 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:45.017606    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk" event={"ID":"20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6","Type":"ContainerDied","Data":"c51768794361b00da3dbc50791710842b2c2f3648eb651176b77d07bff023358"}
Jul 13 19:11:45.018042 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:45.017688    2569 scope.go:117] "RemoveContainer" containerID="74d9ad2057e100f5208e1b5075c9f1daefa590caecfd5713350765f6a5e2904b"
Jul 13 19:11:45.018042 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:45.017876    2569 scope.go:117] "RemoveContainer" containerID="c51768794361b00da3dbc50791710842b2c2f3648eb651176b77d07bff023358"
Jul 13 19:11:45.018221 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:11:45.018129    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-jgdsk_odh-ai-gateway-infra(20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6"
Jul 13 19:11:46.030264 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:46.030236    2569 scope.go:117] "RemoveContainer" containerID="c51768794361b00da3dbc50791710842b2c2f3648eb651176b77d07bff023358"
Jul 13 19:11:46.030671 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:11:46.030425    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-jgdsk_odh-ai-gateway-infra(20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6"
Jul 13 19:11:47.737389 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:47.737356    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:11:47.924135 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:47.924102    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd"]
Jul 13 19:11:47.930025 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:47.930004    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd"
Jul 13 19:11:47.935948 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:47.935919    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd"]
Jul 13 19:11:47.982572 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:47.982544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5585bf3c-8ba4-4244-93fb-790cfc3019d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd\" (UID: \"5585bf3c-8ba4-4244-93fb-790cfc3019d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd"
Jul 13 19:11:48.083396 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:48.083309    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5585bf3c-8ba4-4244-93fb-790cfc3019d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd\" (UID: \"5585bf3c-8ba4-4244-93fb-790cfc3019d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd"
Jul 13 19:11:48.083969 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:48.083950    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5585bf3c-8ba4-4244-93fb-790cfc3019d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd\" (UID: \"5585bf3c-8ba4-4244-93fb-790cfc3019d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd"
Jul 13 19:11:48.242638 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:48.242599    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd"
Jul 13 19:11:48.383032 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:48.383005    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd"]
Jul 13 19:11:48.384713 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:11:48.384687    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5585bf3c_8ba4_4244_93fb_790cfc3019d7.slice/crio-996a384363434e063df1e6725b674a2f7e20b0e903ff6b3be886929db3969067 WatchSource:0}: Error finding container 996a384363434e063df1e6725b674a2f7e20b0e903ff6b3be886929db3969067: Status 404 returned error can't find the container with id 996a384363434e063df1e6725b674a2f7e20b0e903ff6b3be886929db3969067
Jul 13 19:11:49.051400 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:49.051365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd" event={"ID":"5585bf3c-8ba4-4244-93fb-790cfc3019d7","Type":"ContainerStarted","Data":"5e13fc8ecb7aec33a7f1fc596d69c5030532cc2cbe9d248f823d5be3ae767852"}
Jul 13 19:11:49.051799 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:49.051418    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd" event={"ID":"5585bf3c-8ba4-4244-93fb-790cfc3019d7","Type":"ContainerStarted","Data":"996a384363434e063df1e6725b674a2f7e20b0e903ff6b3be886929db3969067"}
Jul 13 19:11:49.074095 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:49.074048    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd" podStartSLOduration=2.074034105 podStartE2EDuration="2.074034105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:11:49.071538245 +0000 UTC m=+2390.202826706" watchObservedRunningTime="2026-07-13 19:11:49.074034105 +0000 UTC m=+2390.205322621"
Jul 13 19:11:50.057931 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:50.057896    2569 generic.go:356] "Generic (PLEG): container finished" podID="5585bf3c-8ba4-4244-93fb-790cfc3019d7" containerID="5e13fc8ecb7aec33a7f1fc596d69c5030532cc2cbe9d248f823d5be3ae767852" exitCode=0
Jul 13 19:11:50.058342 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:50.057991    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd" event={"ID":"5585bf3c-8ba4-4244-93fb-790cfc3019d7","Type":"ContainerDied","Data":"5e13fc8ecb7aec33a7f1fc596d69c5030532cc2cbe9d248f823d5be3ae767852"}
Jul 13 19:11:51.198864 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:51.198840    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd"
Jul 13 19:11:51.210012 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:51.209984    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5585bf3c-8ba4-4244-93fb-790cfc3019d7-maas-api-service-ca\") pod \"5585bf3c-8ba4-4244-93fb-790cfc3019d7\" (UID: \"5585bf3c-8ba4-4244-93fb-790cfc3019d7\") "
Jul 13 19:11:51.210334 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:51.210311    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5585bf3c-8ba4-4244-93fb-790cfc3019d7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5585bf3c-8ba4-4244-93fb-790cfc3019d7" (UID: "5585bf3c-8ba4-4244-93fb-790cfc3019d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:11:51.311568 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:51.311517    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5585bf3c-8ba4-4244-93fb-790cfc3019d7-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:11:52.069698 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:52.069665    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd"
Jul 13 19:11:52.069698 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:52.069688    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd" event={"ID":"5585bf3c-8ba4-4244-93fb-790cfc3019d7","Type":"ContainerDied","Data":"996a384363434e063df1e6725b674a2f7e20b0e903ff6b3be886929db3969067"}
Jul 13 19:11:52.069908 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:52.069762    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="996a384363434e063df1e6725b674a2f7e20b0e903ff6b3be886929db3969067"
Jul 13 19:11:58.344830 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:58.344797    2569 scope.go:117] "RemoveContainer" containerID="c51768794361b00da3dbc50791710842b2c2f3648eb651176b77d07bff023358"
Jul 13 19:11:58.950408 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:58.950372    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"]
Jul 13 19:11:58.950725 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:58.950671    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x" podUID="b05c6fab-9561-484f-a452-eaf0d1d6ea48" containerName="maas-api" containerID="cri-o://2d682289b4a3e87a86cbf79f6a41646ca34e2c5fb532f78b233dfb1ecf88b080" gracePeriod=30
Jul 13 19:11:59.112636 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.112605    2569 generic.go:356] "Generic (PLEG): container finished" podID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerID="8f28dd64b7a6953af99c0569260139f70a2e65ea1f996e5c483083f1ecf1038c" exitCode=6
Jul 13 19:11:59.112832 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.112697    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk" event={"ID":"20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6","Type":"ContainerDied","Data":"8f28dd64b7a6953af99c0569260139f70a2e65ea1f996e5c483083f1ecf1038c"}
Jul 13 19:11:59.112832 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.112767    2569 scope.go:117] "RemoveContainer" containerID="c51768794361b00da3dbc50791710842b2c2f3648eb651176b77d07bff023358"
Jul 13 19:11:59.113110 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.113091    2569 scope.go:117] "RemoveContainer" containerID="8f28dd64b7a6953af99c0569260139f70a2e65ea1f996e5c483083f1ecf1038c"
Jul 13 19:11:59.113382 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:11:59.113355    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-jgdsk_odh-ai-gateway-infra(20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6"
Jul 13 19:11:59.114396 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.114374    2569 generic.go:356] "Generic (PLEG): container finished" podID="b05c6fab-9561-484f-a452-eaf0d1d6ea48" containerID="2d682289b4a3e87a86cbf79f6a41646ca34e2c5fb532f78b233dfb1ecf88b080" exitCode=0
Jul 13 19:11:59.114483 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.114449    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x" event={"ID":"b05c6fab-9561-484f-a452-eaf0d1d6ea48","Type":"ContainerDied","Data":"2d682289b4a3e87a86cbf79f6a41646ca34e2c5fb532f78b233dfb1ecf88b080"}
Jul 13 19:11:59.207479 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.207420    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:11:59.280559 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.280524    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05c6fab-9561-484f-a452-eaf0d1d6ea48-maas-api-tls\") pod \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") "
Jul 13 19:11:59.280788 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.280633    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24nzc\" (UniqueName: \"kubernetes.io/projected/b05c6fab-9561-484f-a452-eaf0d1d6ea48-kube-api-access-24nzc\") pod \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") "
Jul 13 19:11:59.280788 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.280703    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b05c6fab-9561-484f-a452-eaf0d1d6ea48-openshift-service-ca\") pod \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\" (UID: \"b05c6fab-9561-484f-a452-eaf0d1d6ea48\") "
Jul 13 19:11:59.281095 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.281068    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b05c6fab-9561-484f-a452-eaf0d1d6ea48-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b05c6fab-9561-484f-a452-eaf0d1d6ea48" (UID: "b05c6fab-9561-484f-a452-eaf0d1d6ea48"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:11:59.282874 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.282851    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b05c6fab-9561-484f-a452-eaf0d1d6ea48-kube-api-access-24nzc" (OuterVolumeSpecName: "kube-api-access-24nzc") pod "b05c6fab-9561-484f-a452-eaf0d1d6ea48" (UID: "b05c6fab-9561-484f-a452-eaf0d1d6ea48"). InnerVolumeSpecName "kube-api-access-24nzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:11:59.282874 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.282862    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b05c6fab-9561-484f-a452-eaf0d1d6ea48-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b05c6fab-9561-484f-a452-eaf0d1d6ea48" (UID: "b05c6fab-9561-484f-a452-eaf0d1d6ea48"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:11:59.382139 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.382108    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05c6fab-9561-484f-a452-eaf0d1d6ea48-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:11:59.382139 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.382139    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24nzc\" (UniqueName: \"kubernetes.io/projected/b05c6fab-9561-484f-a452-eaf0d1d6ea48-kube-api-access-24nzc\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:11:59.382469 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.382155    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b05c6fab-9561-484f-a452-eaf0d1d6ea48-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:11:59.465573 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.465493    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 19:11:59.472328 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:11:59.472307    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 19:12:00.123254 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:00.123222    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"
Jul 13 19:12:00.123431 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:00.123257    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x" event={"ID":"b05c6fab-9561-484f-a452-eaf0d1d6ea48","Type":"ContainerDied","Data":"45044b69c15b4b4a5a9b7a35120ab30d32ee4499d09bf262dbd97e2e8af35a5a"}
Jul 13 19:12:00.123431 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:00.123318    2569 scope.go:117] "RemoveContainer" containerID="2d682289b4a3e87a86cbf79f6a41646ca34e2c5fb532f78b233dfb1ecf88b080"
Jul 13 19:12:00.141137 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:00.141101    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"]
Jul 13 19:12:00.145694 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:00.145672    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"]
Jul 13 19:12:00.147961 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:00.147942    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e196f1-6bfc49f487-ddn4x"]
Jul 13 19:12:00.279337 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:00.279314    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"
Jul 13 19:12:00.392180 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:00.392094    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6-maas-api-service-ca\") pod \"20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6\" (UID: \"20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6\") "
Jul 13 19:12:00.392548 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:00.392483    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" (UID: "20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:12:00.493294 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:00.493258    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:01.130317 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:01.130277    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk" event={"ID":"20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6","Type":"ContainerDied","Data":"46b6c646b37320e654d048cd8363fd917ce2fec9d01fdc1dd50f40abc3ad3ef4"}
Jul 13 19:12:01.130317 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:01.130313    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"
Jul 13 19:12:01.130561 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:01.130350    2569 scope.go:117] "RemoveContainer" containerID="8f28dd64b7a6953af99c0569260139f70a2e65ea1f996e5c483083f1ecf1038c"
Jul 13 19:12:01.152897 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:01.152875    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"]
Jul 13 19:12:01.163127 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:01.163103    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-jgdsk"]
Jul 13 19:12:01.350025 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:01.349989    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" path="/var/lib/kubelet/pods/20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6/volumes"
Jul 13 19:12:01.350481 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:01.350465    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b05c6fab-9561-484f-a452-eaf0d1d6ea48" path="/var/lib/kubelet/pods/b05c6fab-9561-484f-a452-eaf0d1d6ea48/volumes"
Jul 13 19:12:04.639991 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:04.639951    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:12:13.704166 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:13.704128    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"]
Jul 13 19:12:13.704710 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:13.704343    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy" containerID="cri-o://26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783" gracePeriod=30
Jul 13 19:12:14.093347 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093310    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q"]
Jul 13 19:12:14.093678 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093639    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5585bf3c-8ba4-4244-93fb-790cfc3019d7" containerName="revoke-keys"
Jul 13 19:12:14.093678 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093665    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5585bf3c-8ba4-4244-93fb-790cfc3019d7" containerName="revoke-keys"
Jul 13 19:12:14.093678 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093675    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b05c6fab-9561-484f-a452-eaf0d1d6ea48" containerName="maas-api"
Jul 13 19:12:14.093678 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093680    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b05c6fab-9561-484f-a452-eaf0d1d6ea48" containerName="maas-api"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093694    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerName="revoke-keys"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093700    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerName="revoke-keys"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093707    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerName="revoke-keys"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093712    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerName="revoke-keys"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093723    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerName="revoke-keys"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093729    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerName="revoke-keys"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093785    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerName="revoke-keys"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093792    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerName="revoke-keys"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093798    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5585bf3c-8ba4-4244-93fb-790cfc3019d7" containerName="revoke-keys"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093803    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b05c6fab-9561-484f-a452-eaf0d1d6ea48" containerName="maas-api"
Jul 13 19:12:14.093825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.093809    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="20bdaf22-4e8d-4be1-bb7c-53e08d7b25f6" containerName="revoke-keys"
Jul 13 19:12:14.097056 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.097040    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q"
Jul 13 19:12:14.099524 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.099506    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:12:14.105291 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.105264    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q"]
Jul 13 19:12:14.214802 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.214759    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q\" (UID: \"427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q"
Jul 13 19:12:14.315871 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.315832    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q\" (UID: \"427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q"
Jul 13 19:12:14.316515 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.316489    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q\" (UID: \"427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q"
Jul 13 19:12:14.408762 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.408644    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q"
Jul 13 19:12:14.761757 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.761728    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q"]
Jul 13 19:12:14.763450 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:12:14.763421    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod427d5b7f_58d8_4ef4_9ea2_0cef94b5cd29.slice/crio-ae9b6582ee3e6c05ecf543f07ab571f26f38377b533cd924a47f3f7add8d2e67 WatchSource:0}: Error finding container ae9b6582ee3e6c05ecf543f07ab571f26f38377b533cd924a47f3f7add8d2e67: Status 404 returned error can't find the container with id ae9b6582ee3e6c05ecf543f07ab571f26f38377b533cd924a47f3f7add8d2e67
Jul 13 19:12:14.945246 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.945206    2569 patch_prober.go:28] interesting pod/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 13 19:12:14.945435 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:14.945277    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:12:15.208512 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:15.208418    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q" event={"ID":"427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29","Type":"ContainerStarted","Data":"1cb0a77ed68b34b249ef1268187f49bdc70b3a65374e22106a255178b9e12403"}
Jul 13 19:12:15.208512 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:15.208472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q" event={"ID":"427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29","Type":"ContainerStarted","Data":"ae9b6582ee3e6c05ecf543f07ab571f26f38377b533cd924a47f3f7add8d2e67"}
Jul 13 19:12:15.223792 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:15.223746    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q" podStartSLOduration=1.223731766 podStartE2EDuration="1.223731766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:12:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:12:15.222562137 +0000 UTC m=+2416.353850597" watchObservedRunningTime="2026-07-13 19:12:15.223731766 +0000 UTC m=+2416.355020246"
Jul 13 19:12:16.215534 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:16.215500    2569 generic.go:356] "Generic (PLEG): container finished" podID="427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29" containerID="1cb0a77ed68b34b249ef1268187f49bdc70b3a65374e22106a255178b9e12403" exitCode=0
Jul 13 19:12:16.215940 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:16.215587    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q" event={"ID":"427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29","Type":"ContainerDied","Data":"1cb0a77ed68b34b249ef1268187f49bdc70b3a65374e22106a255178b9e12403"}
Jul 13 19:12:17.355053 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:17.355029    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q"
Jul 13 19:12:17.444760 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:17.444729    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29-maas-api-service-ca\") pod \"427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29\" (UID: \"427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29\") "
Jul 13 19:12:17.445077 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:17.445055    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29" (UID: "427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:12:17.545448 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:17.545420    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:18.226563 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:18.226537    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q"
Jul 13 19:12:18.226563 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:18.226542    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q" event={"ID":"427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29","Type":"ContainerDied","Data":"ae9b6582ee3e6c05ecf543f07ab571f26f38377b533cd924a47f3f7add8d2e67"}
Jul 13 19:12:18.226815 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:18.226590    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae9b6582ee3e6c05ecf543f07ab571f26f38377b533cd924a47f3f7add8d2e67"
Jul 13 19:12:18.981214 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:18.981191    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:12:19.057365 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057328    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-socket\") pod \"6410559f-81bd-48ca-911d-da11c4d52a3f\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") "
Jul 13 19:12:19.057548 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057415    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-credential-socket\") pod \"6410559f-81bd-48ca-911d-da11c4d52a3f\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") "
Jul 13 19:12:19.057548 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057450    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vlv9\" (UniqueName: \"kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-kube-api-access-7vlv9\") pod \"6410559f-81bd-48ca-911d-da11c4d52a3f\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") "
Jul 13 19:12:19.057548 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057478    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-podinfo\") pod \"6410559f-81bd-48ca-911d-da11c4d52a3f\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") "
Jul 13 19:12:19.057548 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057502    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-token\") pod \"6410559f-81bd-48ca-911d-da11c4d52a3f\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") "
Jul 13 19:12:19.057548 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057545    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6410559f-81bd-48ca-911d-da11c4d52a3f-istiod-ca-cert\") pod \"6410559f-81bd-48ca-911d-da11c4d52a3f\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") "
Jul 13 19:12:19.057846 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057571    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-data\") pod \"6410559f-81bd-48ca-911d-da11c4d52a3f\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") "
Jul 13 19:12:19.057846 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057602    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-envoy\") pod \"6410559f-81bd-48ca-911d-da11c4d52a3f\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") "
Jul 13 19:12:19.057846 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057599    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6410559f-81bd-48ca-911d-da11c4d52a3f" (UID: "6410559f-81bd-48ca-911d-da11c4d52a3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:12:19.057846 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057729    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-certs\") pod \"6410559f-81bd-48ca-911d-da11c4d52a3f\" (UID: \"6410559f-81bd-48ca-911d-da11c4d52a3f\") "
Jul 13 19:12:19.057846 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.057766    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6410559f-81bd-48ca-911d-da11c4d52a3f" (UID: "6410559f-81bd-48ca-911d-da11c4d52a3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:12:19.058102 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.058019    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:19.058102 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.058040    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-credential-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:19.058102 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.058040    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6410559f-81bd-48ca-911d-da11c4d52a3f" (UID: "6410559f-81bd-48ca-911d-da11c4d52a3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:12:19.058287 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.058247    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6410559f-81bd-48ca-911d-da11c4d52a3f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6410559f-81bd-48ca-911d-da11c4d52a3f" (UID: "6410559f-81bd-48ca-911d-da11c4d52a3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:12:19.058573 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.058503    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-data" (OuterVolumeSpecName: "istio-data") pod "6410559f-81bd-48ca-911d-da11c4d52a3f" (UID: "6410559f-81bd-48ca-911d-da11c4d52a3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:12:19.060184 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.060146    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-token" (OuterVolumeSpecName: "istio-token") pod "6410559f-81bd-48ca-911d-da11c4d52a3f" (UID: "6410559f-81bd-48ca-911d-da11c4d52a3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:12:19.060310 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.060236    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6410559f-81bd-48ca-911d-da11c4d52a3f" (UID: "6410559f-81bd-48ca-911d-da11c4d52a3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 19:12:19.060375 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.060315    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6410559f-81bd-48ca-911d-da11c4d52a3f" (UID: "6410559f-81bd-48ca-911d-da11c4d52a3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:12:19.060375 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.060325    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-kube-api-access-7vlv9" (OuterVolumeSpecName: "kube-api-access-7vlv9") pod "6410559f-81bd-48ca-911d-da11c4d52a3f" (UID: "6410559f-81bd-48ca-911d-da11c4d52a3f"). InnerVolumeSpecName "kube-api-access-7vlv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:12:19.159211 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.159179    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vlv9\" (UniqueName: \"kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-kube-api-access-7vlv9\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:19.159211 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.159206    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-podinfo\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:19.159211 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.159216    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-token\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:19.159447 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.159225    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6410559f-81bd-48ca-911d-da11c4d52a3f-istiod-ca-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:19.159447 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.159233    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-data\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:19.159447 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.159240    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-istio-envoy\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:19.159447 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.159248    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6410559f-81bd-48ca-911d-da11c4d52a3f-workload-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:19.233495 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.233458    2569 generic.go:356] "Generic (PLEG): container finished" podID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerID="26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783" exitCode=0
Jul 13 19:12:19.233680 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.233511    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"
Jul 13 19:12:19.233680 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.233507    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" event={"ID":"6410559f-81bd-48ca-911d-da11c4d52a3f","Type":"ContainerDied","Data":"26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783"}
Jul 13 19:12:19.233680 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.233676    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv" event={"ID":"6410559f-81bd-48ca-911d-da11c4d52a3f","Type":"ContainerDied","Data":"4fe4e28aac634d443ee2e79bd1f7b8bfc7abbc8b4710c19bf9e37966a244982d"}
Jul 13 19:12:19.233784 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.233702    2569 scope.go:117] "RemoveContainer" containerID="26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783"
Jul 13 19:12:19.243358 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.243342    2569 scope.go:117] "RemoveContainer" containerID="8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab"
Jul 13 19:12:19.251954 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.251933    2569 scope.go:117] "RemoveContainer" containerID="26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783"
Jul 13 19:12:19.252221 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:12:19.252202    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783\": container with ID starting with 26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783 not found: ID does not exist" containerID="26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783"
Jul 13 19:12:19.252261 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.252230    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783"} err="failed to get container status \"26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783\": rpc error: code = NotFound desc = could not find container \"26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783\": container with ID starting with 26ae13df64c195da5386191266336c68ecb036e3bb4ae1b7fa74f59f50f0f783 not found: ID does not exist"
Jul 13 19:12:19.252261 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.252247    2569 scope.go:117] "RemoveContainer" containerID="8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab"
Jul 13 19:12:19.252480 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:12:19.252464    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab\": container with ID starting with 8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab not found: ID does not exist" containerID="8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab"
Jul 13 19:12:19.252524 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.252482    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab"} err="failed to get container status \"8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab\": rpc error: code = NotFound desc = could not find container \"8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab\": container with ID starting with 8be3b1e8904bc28baf0fb4bafe18272dd845fffbdf15e92044258c65a185fcab not found: ID does not exist"
Jul 13 19:12:19.258179 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.258156    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"]
Jul 13 19:12:19.262986 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.262964    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-e196f1-openshift-default-6fdd78b479-kghzv"]
Jul 13 19:12:19.350560 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:19.350520    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" path="/var/lib/kubelet/pods/6410559f-81bd-48ca-911d-da11c4d52a3f/volumes"
Jul 13 19:12:25.019871 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.019839    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"]
Jul 13 19:12:25.027497 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.027451    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2" podUID="a672e79c-fb0c-4155-9f22-da5af2b84efe" containerName="maas-api" containerID="cri-o://4d65db44428bfcc2649749ec344a56a36c580fde6fb1c8ef59374cc11252da39" gracePeriod=30
Jul 13 19:12:25.269985 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.269908    2569 generic.go:356] "Generic (PLEG): container finished" podID="a672e79c-fb0c-4155-9f22-da5af2b84efe" containerID="4d65db44428bfcc2649749ec344a56a36c580fde6fb1c8ef59374cc11252da39" exitCode=0
Jul 13 19:12:25.269985 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.269936    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2" event={"ID":"a672e79c-fb0c-4155-9f22-da5af2b84efe","Type":"ContainerDied","Data":"4d65db44428bfcc2649749ec344a56a36c580fde6fb1c8ef59374cc11252da39"}
Jul 13 19:12:25.270144 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.270003    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2" event={"ID":"a672e79c-fb0c-4155-9f22-da5af2b84efe","Type":"ContainerDied","Data":"8e783dc0cbd3290bf4f3e40fef9fca0d97d1cd0d10719399360ec980ec5a5c04"}
Jul 13 19:12:25.270144 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.270027    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e783dc0cbd3290bf4f3e40fef9fca0d97d1cd0d10719399360ec980ec5a5c04"
Jul 13 19:12:25.276541 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.276523    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:12:25.413697 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.413666    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a672e79c-fb0c-4155-9f22-da5af2b84efe-openshift-service-ca\") pod \"a672e79c-fb0c-4155-9f22-da5af2b84efe\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") "
Jul 13 19:12:25.413881 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.413704    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x97vc\" (UniqueName: \"kubernetes.io/projected/a672e79c-fb0c-4155-9f22-da5af2b84efe-kube-api-access-x97vc\") pod \"a672e79c-fb0c-4155-9f22-da5af2b84efe\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") "
Jul 13 19:12:25.413881 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.413723    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a672e79c-fb0c-4155-9f22-da5af2b84efe-maas-api-tls\") pod \"a672e79c-fb0c-4155-9f22-da5af2b84efe\" (UID: \"a672e79c-fb0c-4155-9f22-da5af2b84efe\") "
Jul 13 19:12:25.414017 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.413988    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a672e79c-fb0c-4155-9f22-da5af2b84efe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a672e79c-fb0c-4155-9f22-da5af2b84efe" (UID: "a672e79c-fb0c-4155-9f22-da5af2b84efe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:12:25.415887 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.415865    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a672e79c-fb0c-4155-9f22-da5af2b84efe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a672e79c-fb0c-4155-9f22-da5af2b84efe" (UID: "a672e79c-fb0c-4155-9f22-da5af2b84efe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:12:25.416049 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.416032    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a672e79c-fb0c-4155-9f22-da5af2b84efe-kube-api-access-x97vc" (OuterVolumeSpecName: "kube-api-access-x97vc") pod "a672e79c-fb0c-4155-9f22-da5af2b84efe" (UID: "a672e79c-fb0c-4155-9f22-da5af2b84efe"). InnerVolumeSpecName "kube-api-access-x97vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:12:25.514414 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.514381    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a672e79c-fb0c-4155-9f22-da5af2b84efe-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:25.514414 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.514411    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x97vc\" (UniqueName: \"kubernetes.io/projected/a672e79c-fb0c-4155-9f22-da5af2b84efe-kube-api-access-x97vc\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:25.514613 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:25.514422    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a672e79c-fb0c-4155-9f22-da5af2b84efe-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:26.276789 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:26.276751    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"
Jul 13 19:12:26.298503 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:26.298475    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"]
Jul 13 19:12:26.305171 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:26.305146    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5ffdec-9bb4d8797-6hbd2"]
Jul 13 19:12:27.350413 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:27.350377    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a672e79c-fb0c-4155-9f22-da5af2b84efe" path="/var/lib/kubelet/pods/a672e79c-fb0c-4155-9f22-da5af2b84efe/volumes"
Jul 13 19:12:28.243628 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:28.243597    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:12:32.202095 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.202057    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"]
Jul 13 19:12:32.202477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.202374    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29" containerName="revoke-keys"
Jul 13 19:12:32.202477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.202384    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29" containerName="revoke-keys"
Jul 13 19:12:32.202477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.202393    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a672e79c-fb0c-4155-9f22-da5af2b84efe" containerName="maas-api"
Jul 13 19:12:32.202477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.202398    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a672e79c-fb0c-4155-9f22-da5af2b84efe" containerName="maas-api"
Jul 13 19:12:32.202477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.202413    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy"
Jul 13 19:12:32.202477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.202419    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy"
Jul 13 19:12:32.202702 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.202500    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy"
Jul 13 19:12:32.202702 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.202508    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29" containerName="revoke-keys"
Jul 13 19:12:32.202702 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.202516    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a672e79c-fb0c-4155-9f22-da5af2b84efe" containerName="maas-api"
Jul 13 19:12:32.206626 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.206607    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"
Jul 13 19:12:32.209168 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.209146    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:12:32.209281 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.209191    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:12:32.214510 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.214485    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"]
Jul 13 19:12:32.372553 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.372515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e69d6b-81e2-46e4-badb-bb1b28ca662b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-wv8jh\" (UID: \"99e69d6b-81e2-46e4-badb-bb1b28ca662b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"
Jul 13 19:12:32.473626 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.473533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e69d6b-81e2-46e4-badb-bb1b28ca662b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-wv8jh\" (UID: \"99e69d6b-81e2-46e4-badb-bb1b28ca662b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"
Jul 13 19:12:32.474217 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.474193    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e69d6b-81e2-46e4-badb-bb1b28ca662b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-wv8jh\" (UID: \"99e69d6b-81e2-46e4-badb-bb1b28ca662b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"
Jul 13 19:12:32.517593 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.517565    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"
Jul 13 19:12:32.658417 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:32.658391    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"]
Jul 13 19:12:32.660123 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:12:32.660096    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e69d6b_81e2_46e4_badb_bb1b28ca662b.slice/crio-71c7a990f1c328c4408f833a5576443f855a5dc22bb8c56e81a9809ffb22d0b5 WatchSource:0}: Error finding container 71c7a990f1c328c4408f833a5576443f855a5dc22bb8c56e81a9809ffb22d0b5: Status 404 returned error can't find the container with id 71c7a990f1c328c4408f833a5576443f855a5dc22bb8c56e81a9809ffb22d0b5
Jul 13 19:12:33.315575 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:33.315538    2569 generic.go:356] "Generic (PLEG): container finished" podID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerID="c951ebb91798b9a464b52a07bdbce8c294cdf8bf42943ade60697bcd683ee569" exitCode=6
Jul 13 19:12:33.315960 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:33.315625    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh" event={"ID":"99e69d6b-81e2-46e4-badb-bb1b28ca662b","Type":"ContainerDied","Data":"c951ebb91798b9a464b52a07bdbce8c294cdf8bf42943ade60697bcd683ee569"}
Jul 13 19:12:33.315960 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:33.315692    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh" event={"ID":"99e69d6b-81e2-46e4-badb-bb1b28ca662b","Type":"ContainerStarted","Data":"71c7a990f1c328c4408f833a5576443f855a5dc22bb8c56e81a9809ffb22d0b5"}
Jul 13 19:12:33.315960 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:33.315802    2569 scope.go:117] "RemoveContainer" containerID="c951ebb91798b9a464b52a07bdbce8c294cdf8bf42943ade60697bcd683ee569"
Jul 13 19:12:34.322850 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:34.322754    2569 generic.go:356] "Generic (PLEG): container finished" podID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerID="03fdd5c3a5d159d3638228c8354d31369faaf1abf7ba7098a297ad5a8c21e24c" exitCode=6
Jul 13 19:12:34.322850 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:34.322837    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh" event={"ID":"99e69d6b-81e2-46e4-badb-bb1b28ca662b","Type":"ContainerDied","Data":"03fdd5c3a5d159d3638228c8354d31369faaf1abf7ba7098a297ad5a8c21e24c"}
Jul 13 19:12:34.323342 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:34.322891    2569 scope.go:117] "RemoveContainer" containerID="c951ebb91798b9a464b52a07bdbce8c294cdf8bf42943ade60697bcd683ee569"
Jul 13 19:12:34.323342 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:34.323057    2569 scope.go:117] "RemoveContainer" containerID="03fdd5c3a5d159d3638228c8354d31369faaf1abf7ba7098a297ad5a8c21e24c"
Jul 13 19:12:34.323342 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:12:34.323309    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-wv8jh_odh-ai-gateway-infra(99e69d6b-81e2-46e4-badb-bb1b28ca662b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b"
Jul 13 19:12:35.330899 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:35.330872    2569 scope.go:117] "RemoveContainer" containerID="03fdd5c3a5d159d3638228c8354d31369faaf1abf7ba7098a297ad5a8c21e24c"
Jul 13 19:12:35.331258 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:12:35.331057    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-wv8jh_odh-ai-gateway-infra(99e69d6b-81e2-46e4-badb-bb1b28ca662b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b"
Jul 13 19:12:40.837108 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.837072    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"]
Jul 13 19:12:40.837470 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.837409    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy"
Jul 13 19:12:40.837470 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.837419    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy"
Jul 13 19:12:40.837470 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.837471    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6410559f-81bd-48ca-911d-da11c4d52a3f" containerName="istio-proxy"
Jul 13 19:12:40.841641 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.841613    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:40.844314 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.844296    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-7b883e-openshift-default-dockercfg-zw8q7\""
Jul 13 19:12:40.853609 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.853584    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"]
Jul 13 19:12:40.946279 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.946240    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee25864f-6bd9-4d16-8310-7c5f794df9da-istiod-ca-cert\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:40.946437 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.946287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-socket\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:40.946437 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.946323    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-token\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:40.946437 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.946339    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-podinfo\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:40.946437 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.946360    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-certs\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:40.946437 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.946383    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-data\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:40.946437 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.946408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blvb5\" (UniqueName: \"kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-kube-api-access-blvb5\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:40.946437 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.946427    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-envoy\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:40.946692 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:40.946482    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-credential-socket\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047302 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047266    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-credential-socket\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee25864f-6bd9-4d16-8310-7c5f794df9da-istiod-ca-cert\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047356    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-socket\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047390    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-token\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047413    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-podinfo\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047444    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-certs\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047467    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-data\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047844 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blvb5\" (UniqueName: \"kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-kube-api-access-blvb5\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047844 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-envoy\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047844 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047784    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-credential-socket\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047994 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047842    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-data\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.047994 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.047900    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-certs\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.048115 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.048071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-socket\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.048207 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.048184    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee25864f-6bd9-4d16-8310-7c5f794df9da-istiod-ca-cert\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.050113 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.050080    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-envoy\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.050223 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.050156    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-podinfo\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.058529 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.058504    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-token\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.058631 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.058583    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blvb5\" (UniqueName: \"kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-kube-api-access-blvb5\") pod \"e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") " pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.153466 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.153371    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:41.306908 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.306855    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:12:41.307273 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.307242    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:12:41.307374 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.307307    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:12:41.307855 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.307809    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"]
Jul 13 19:12:41.362998 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:41.362968    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" event={"ID":"ee25864f-6bd9-4d16-8310-7c5f794df9da","Type":"ContainerStarted","Data":"54997a26826a4e58a2c58e252ebcc34f4d28c08c1f52e3821c9df2972a8d8cde"}
Jul 13 19:12:42.369999 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:42.369961    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" event={"ID":"ee25864f-6bd9-4d16-8310-7c5f794df9da","Type":"ContainerStarted","Data":"efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c"}
Jul 13 19:12:42.394604 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:42.394555    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" podStartSLOduration=2.394533562 podStartE2EDuration="2.394533562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:12:42.392343716 +0000 UTC m=+2443.523632180" watchObservedRunningTime="2026-07-13 19:12:42.394533562 +0000 UTC m=+2443.525822108"
Jul 13 19:12:43.153551 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:43.153519    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:43.158413 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:43.158389    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:43.374886 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:43.374851    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:43.375950 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:43.375931    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:44.468281 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.468241    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"]
Jul 13 19:12:44.472059 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.472035    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:44.476245 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.476224    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xfcfp\""
Jul 13 19:12:44.476381 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.476296    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-7b883e-serving-cert\""
Jul 13 19:12:44.477030 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.477013    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 13 19:12:44.487840 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.487813    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"]
Jul 13 19:12:44.578700 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.578666    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:44.578910 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.578739    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79hxv\" (UniqueName: \"kubernetes.io/projected/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-kube-api-access-79hxv\") pod \"maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:44.578994 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.578975    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:44.680353 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.680314    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:44.680353 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.680349    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:44.680628 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:12:44.680449    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-serving-cert: secret "maas-api-e2e-shared-a-7b883e-serving-cert" not found
Jul 13 19:12:44.680628 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.680488    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79hxv\" (UniqueName: \"kubernetes.io/projected/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-kube-api-access-79hxv\") pod \"maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:44.680628 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:12:44.680511    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-maas-api-tls podName:fb2a1046-a2cf-44a2-bbdf-0453c91e58dc nodeName:}" failed. No retries permitted until 2026-07-13 19:12:45.180492373 +0000 UTC m=+2446.311780812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-maas-api-tls") pod "maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl" (UID: "fb2a1046-a2cf-44a2-bbdf-0453c91e58dc") : secret "maas-api-e2e-shared-a-7b883e-serving-cert" not found
Jul 13 19:12:44.681063 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.681043    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:44.693307 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:44.693280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79hxv\" (UniqueName: \"kubernetes.io/projected/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-kube-api-access-79hxv\") pod \"maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:45.184450 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:45.184415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:45.187196 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:45.187166    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:45.383671 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:45.383628    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:45.543217 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:45.543171    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"]
Jul 13 19:12:45.544889 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:12:45.544859    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2a1046_a2cf_44a2_bbdf_0453c91e58dc.slice/crio-cc9747c839a1356352a50c5ad699ed0a62da5d786907822b411f8f8c79ed0865 WatchSource:0}: Error finding container cc9747c839a1356352a50c5ad699ed0a62da5d786907822b411f8f8c79ed0865: Status 404 returned error can't find the container with id cc9747c839a1356352a50c5ad699ed0a62da5d786907822b411f8f8c79ed0865
Jul 13 19:12:46.393040 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:46.392950    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl" event={"ID":"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc","Type":"ContainerStarted","Data":"67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4"}
Jul 13 19:12:46.393040 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:46.393001    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl" event={"ID":"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc","Type":"ContainerStarted","Data":"cc9747c839a1356352a50c5ad699ed0a62da5d786907822b411f8f8c79ed0865"}
Jul 13 19:12:46.393269 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:46.393044    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:46.413206 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:46.413158    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl" podStartSLOduration=1.959254857 podStartE2EDuration="2.413142612s" totalImagesPullingTime="453.887755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:12:46.410867381 +0000 UTC m=+2447.542155841" watchObservedRunningTime="2026-07-13 19:12:46.413142612 +0000 UTC m=+2447.544431072"
Jul 13 19:12:48.345184 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:48.345153    2569 scope.go:117] "RemoveContainer" containerID="03fdd5c3a5d159d3638228c8354d31369faaf1abf7ba7098a297ad5a8c21e24c"
Jul 13 19:12:49.339320 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:49.339283    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:12:49.412061 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:49.412031    2569 generic.go:356] "Generic (PLEG): container finished" podID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerID="4a53231328e2fd1c38bbc6e508427a45b4fdc41cc32cba93c94c6760f1431af3" exitCode=6
Jul 13 19:12:49.412486 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:49.412091    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh" event={"ID":"99e69d6b-81e2-46e4-badb-bb1b28ca662b","Type":"ContainerDied","Data":"4a53231328e2fd1c38bbc6e508427a45b4fdc41cc32cba93c94c6760f1431af3"}
Jul 13 19:12:49.412486 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:49.412139    2569 scope.go:117] "RemoveContainer" containerID="03fdd5c3a5d159d3638228c8354d31369faaf1abf7ba7098a297ad5a8c21e24c"
Jul 13 19:12:49.412486 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:49.412433    2569 scope.go:117] "RemoveContainer" containerID="4a53231328e2fd1c38bbc6e508427a45b4fdc41cc32cba93c94c6760f1431af3"
Jul 13 19:12:49.412709 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:12:49.412675    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-wv8jh_odh-ai-gateway-infra(99e69d6b-81e2-46e4-badb-bb1b28ca662b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b"
Jul 13 19:12:50.445295 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:50.445259    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"]
Jul 13 19:12:50.612685 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:50.612309    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"
Jul 13 19:12:50.632378 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:50.632348    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e69d6b-81e2-46e4-badb-bb1b28ca662b-maas-api-service-ca\") pod \"99e69d6b-81e2-46e4-badb-bb1b28ca662b\" (UID: \"99e69d6b-81e2-46e4-badb-bb1b28ca662b\") "
Jul 13 19:12:50.632883 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:50.632850    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99e69d6b-81e2-46e4-badb-bb1b28ca662b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "99e69d6b-81e2-46e4-badb-bb1b28ca662b" (UID: "99e69d6b-81e2-46e4-badb-bb1b28ca662b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:12:50.733748 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:50.733724    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e69d6b-81e2-46e4-badb-bb1b28ca662b-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:12:51.432237 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:51.432202    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"
Jul 13 19:12:51.432237 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:51.432219    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh" event={"ID":"99e69d6b-81e2-46e4-badb-bb1b28ca662b","Type":"ContainerDied","Data":"71c7a990f1c328c4408f833a5576443f855a5dc22bb8c56e81a9809ffb22d0b5"}
Jul 13 19:12:51.432438 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:51.432279    2569 scope.go:117] "RemoveContainer" containerID="4a53231328e2fd1c38bbc6e508427a45b4fdc41cc32cba93c94c6760f1431af3"
Jul 13 19:12:51.449263 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:51.449241    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"]
Jul 13 19:12:51.456278 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:51.456253    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-wv8jh"]
Jul 13 19:12:52.415634 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.415599    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:12:52.796942 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.796906    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"]
Jul 13 19:12:52.797304 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.797243    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerName="revoke-keys"
Jul 13 19:12:52.797304 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.797254    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerName="revoke-keys"
Jul 13 19:12:52.797304 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.797267    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerName="revoke-keys"
Jul 13 19:12:52.797304 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.797273    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerName="revoke-keys"
Jul 13 19:12:52.797304 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.797283    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerName="revoke-keys"
Jul 13 19:12:52.797304 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.797289    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerName="revoke-keys"
Jul 13 19:12:52.797498 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.797367    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerName="revoke-keys"
Jul 13 19:12:52.797498 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.797379    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerName="revoke-keys"
Jul 13 19:12:52.831907 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.831872    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"]
Jul 13 19:12:52.833450 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.833426    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:52.838206 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.838187    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-e91902-serving-cert\""
Jul 13 19:12:52.853028 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.852999    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcf65a7-f5e6-4b33-8025-a1906106687d-maas-api-tls\") pod \"maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:52.853183 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.853080    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bcf65a7-f5e6-4b33-8025-a1906106687d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:52.853183 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.853134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srrm6\" (UniqueName: \"kubernetes.io/projected/5bcf65a7-f5e6-4b33-8025-a1906106687d-kube-api-access-srrm6\") pod \"maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:52.953847 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.953816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcf65a7-f5e6-4b33-8025-a1906106687d-maas-api-tls\") pod \"maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:52.954028 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.953868    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bcf65a7-f5e6-4b33-8025-a1906106687d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:52.954028 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.953901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srrm6\" (UniqueName: \"kubernetes.io/projected/5bcf65a7-f5e6-4b33-8025-a1906106687d-kube-api-access-srrm6\") pod \"maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:52.954028 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:12:52.953975    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-serving-cert: secret "maas-api-e2e-shared-b-e91902-serving-cert" not found
Jul 13 19:12:52.954137 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:12:52.954058    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bcf65a7-f5e6-4b33-8025-a1906106687d-maas-api-tls podName:5bcf65a7-f5e6-4b33-8025-a1906106687d nodeName:}" failed. No retries permitted until 2026-07-13 19:12:53.454039704 +0000 UTC m=+2454.585328145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5bcf65a7-f5e6-4b33-8025-a1906106687d-maas-api-tls") pod "maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6" (UID: "5bcf65a7-f5e6-4b33-8025-a1906106687d") : secret "maas-api-e2e-shared-b-e91902-serving-cert" not found
Jul 13 19:12:52.954615 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.954596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bcf65a7-f5e6-4b33-8025-a1906106687d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:52.963487 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:52.963464    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srrm6\" (UniqueName: \"kubernetes.io/projected/5bcf65a7-f5e6-4b33-8025-a1906106687d-kube-api-access-srrm6\") pod \"maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:53.350798 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:53.350762    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" path="/var/lib/kubelet/pods/99e69d6b-81e2-46e4-badb-bb1b28ca662b/volumes"
Jul 13 19:12:53.456896 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:53.456845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcf65a7-f5e6-4b33-8025-a1906106687d-maas-api-tls\") pod \"maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:53.459494 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:53.459474    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcf65a7-f5e6-4b33-8025-a1906106687d-maas-api-tls\") pod \"maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:53.748783 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:53.748747    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:53.932625 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:53.932599    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"]
Jul 13 19:12:53.935428 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:12:53.935400    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bcf65a7_f5e6_4b33_8025_a1906106687d.slice/crio-47b5de9508cd959e9be3e1fda2e69219588e4fd5820161120d60ea99b8793dd2 WatchSource:0}: Error finding container 47b5de9508cd959e9be3e1fda2e69219588e4fd5820161120d60ea99b8793dd2: Status 404 returned error can't find the container with id 47b5de9508cd959e9be3e1fda2e69219588e4fd5820161120d60ea99b8793dd2
Jul 13 19:12:54.451661 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:54.451617    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6" event={"ID":"5bcf65a7-f5e6-4b33-8025-a1906106687d","Type":"ContainerStarted","Data":"2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01"}
Jul 13 19:12:54.451822 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:54.451687    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6" event={"ID":"5bcf65a7-f5e6-4b33-8025-a1906106687d","Type":"ContainerStarted","Data":"47b5de9508cd959e9be3e1fda2e69219588e4fd5820161120d60ea99b8793dd2"}
Jul 13 19:12:54.451822 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:54.451737    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:12:54.472939 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:54.472890    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6" podStartSLOduration=2.126792741 podStartE2EDuration="2.472876728s" totalImagesPullingTime="346.083987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:12:54.470159326 +0000 UTC m=+2455.601447786" watchObservedRunningTime="2026-07-13 19:12:54.472876728 +0000 UTC m=+2455.604165188"
Jul 13 19:12:57.471741 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:57.471623    2569 generic.go:356] "Generic (PLEG): container finished" podID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerID="efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c" exitCode=0
Jul 13 19:12:57.471741 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:57.471694    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" event={"ID":"ee25864f-6bd9-4d16-8310-7c5f794df9da","Type":"ContainerDied","Data":"efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c"}
Jul 13 19:12:57.472191 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:57.472070    2569 scope.go:117] "RemoveContainer" containerID="efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c"
Jul 13 19:12:57.473078 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:57.473047    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:12:57.473186 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:57.473132    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:12:57.473186 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:57.473180    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 13 19:12:58.375163 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:58.375135    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:58.479269 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:58.479237    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" event={"ID":"ee25864f-6bd9-4d16-8310-7c5f794df9da","Type":"ContainerStarted","Data":"db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc"}
Jul 13 19:12:58.639302 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:58.639224    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:12:59.153844 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:59.153813    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:12:59.155036 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:59.155015    2569 patch_prober.go:28] interesting pod/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Jul 13 19:12:59.155091 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:12:59.155063    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Jul 13 19:13:00.154377 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:00.154338    2569 patch_prober.go:28] interesting pod/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Jul 13 19:13:00.154805 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:00.154408    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Jul 13 19:13:00.464298 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:00.464221    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:13:01.154329 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:01.154292    2569 patch_prober.go:28] interesting pod/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Jul 13 19:13:01.154509 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:01.154347    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Jul 13 19:13:02.153853 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:02.153816    2569 patch_prober.go:28] interesting pod/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Jul 13 19:13:02.154038 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:02.153875    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Jul 13 19:13:03.157530 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:03.157499    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:13:03.158023 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:03.157910    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:13:03.158674 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:03.158633    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:13:14.238110 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:13:14.238076    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:14:10.142115 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:10.142035    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:14:12.296320 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.296287    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"]
Jul 13 19:14:12.296721 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.296711    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="99e69d6b-81e2-46e4-badb-bb1b28ca662b" containerName="revoke-keys"
Jul 13 19:14:12.299886 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.299871    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"
Jul 13 19:14:12.302532 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.302510    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:14:12.309390 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.309366    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"]
Jul 13 19:14:12.352609 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.352576    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe83dc96-c034-4bde-ad90-26c29ad3d10d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-6x8xg\" (UID: \"fe83dc96-c034-4bde-ad90-26c29ad3d10d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"
Jul 13 19:14:12.453397 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.453365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe83dc96-c034-4bde-ad90-26c29ad3d10d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-6x8xg\" (UID: \"fe83dc96-c034-4bde-ad90-26c29ad3d10d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"
Jul 13 19:14:12.453989 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.453971    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe83dc96-c034-4bde-ad90-26c29ad3d10d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-6x8xg\" (UID: \"fe83dc96-c034-4bde-ad90-26c29ad3d10d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"
Jul 13 19:14:12.611794 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.611713    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"
Jul 13 19:14:12.752863 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.752840    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"]
Jul 13 19:14:12.754555 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:14:12.754530    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe83dc96_c034_4bde_ad90_26c29ad3d10d.slice/crio-9f05126bfc56e94d09f8b3903513140e71a7b35d2a135eab35abb10c0e72f8a0 WatchSource:0}: Error finding container 9f05126bfc56e94d09f8b3903513140e71a7b35d2a135eab35abb10c0e72f8a0: Status 404 returned error can't find the container with id 9f05126bfc56e94d09f8b3903513140e71a7b35d2a135eab35abb10c0e72f8a0
Jul 13 19:14:12.893759 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:12.893675    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg" event={"ID":"fe83dc96-c034-4bde-ad90-26c29ad3d10d","Type":"ContainerStarted","Data":"9f05126bfc56e94d09f8b3903513140e71a7b35d2a135eab35abb10c0e72f8a0"}
Jul 13 19:14:13.900732 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:13.900694    2569 generic.go:356] "Generic (PLEG): container finished" podID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerID="67e5b6cde7bf7b560d011684f802649ead2a51e933ea82d9ba5624e6e22baf50" exitCode=6
Jul 13 19:14:13.901109 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:13.900778    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg" event={"ID":"fe83dc96-c034-4bde-ad90-26c29ad3d10d","Type":"ContainerDied","Data":"67e5b6cde7bf7b560d011684f802649ead2a51e933ea82d9ba5624e6e22baf50"}
Jul 13 19:14:13.901109 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:13.900927    2569 scope.go:117] "RemoveContainer" containerID="67e5b6cde7bf7b560d011684f802649ead2a51e933ea82d9ba5624e6e22baf50"
Jul 13 19:14:14.908232 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:14.908135    2569 generic.go:356] "Generic (PLEG): container finished" podID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerID="0fde59aa8154b09f074e9f214501153eff3277842c478f040d1e513994ac765e" exitCode=6
Jul 13 19:14:14.908607 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:14.908223    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg" event={"ID":"fe83dc96-c034-4bde-ad90-26c29ad3d10d","Type":"ContainerDied","Data":"0fde59aa8154b09f074e9f214501153eff3277842c478f040d1e513994ac765e"}
Jul 13 19:14:14.908607 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:14.908281    2569 scope.go:117] "RemoveContainer" containerID="67e5b6cde7bf7b560d011684f802649ead2a51e933ea82d9ba5624e6e22baf50"
Jul 13 19:14:14.908607 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:14.908407    2569 scope.go:117] "RemoveContainer" containerID="0fde59aa8154b09f074e9f214501153eff3277842c478f040d1e513994ac765e"
Jul 13 19:14:14.908821 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:14:14.908630    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-6x8xg_odh-ai-gateway-infra(fe83dc96-c034-4bde-ad90-26c29ad3d10d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d"
Jul 13 19:14:15.915431 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:15.915402    2569 scope.go:117] "RemoveContainer" containerID="0fde59aa8154b09f074e9f214501153eff3277842c478f040d1e513994ac765e"
Jul 13 19:14:15.915847 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:14:15.915587    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-6x8xg_odh-ai-gateway-infra(fe83dc96-c034-4bde-ad90-26c29ad3d10d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d"
Jul 13 19:14:25.341647 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:25.341615    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:14:26.345087 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:26.345055    2569 scope.go:117] "RemoveContainer" containerID="0fde59aa8154b09f074e9f214501153eff3277842c478f040d1e513994ac765e"
Jul 13 19:14:26.977006 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:26.976970    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg" event={"ID":"fe83dc96-c034-4bde-ad90-26c29ad3d10d","Type":"ContainerStarted","Data":"62357dfad1fd6abb549dd47ca468002d9d425c75924a629fdf16b66f1e770bde"}
Jul 13 19:14:27.010868 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:27.010823    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg" podStartSLOduration=15.010808057 podStartE2EDuration="15.010808057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:14:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:14:27.00837014 +0000 UTC m=+2548.139658600" watchObservedRunningTime="2026-07-13 19:14:27.010808057 +0000 UTC m=+2548.142096517"
Jul 13 19:14:27.983880 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:27.983847    2569 generic.go:356] "Generic (PLEG): container finished" podID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerID="62357dfad1fd6abb549dd47ca468002d9d425c75924a629fdf16b66f1e770bde" exitCode=6
Jul 13 19:14:27.984298 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:27.983890    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg" event={"ID":"fe83dc96-c034-4bde-ad90-26c29ad3d10d","Type":"ContainerDied","Data":"62357dfad1fd6abb549dd47ca468002d9d425c75924a629fdf16b66f1e770bde"}
Jul 13 19:14:27.984298 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:27.983947    2569 scope.go:117] "RemoveContainer" containerID="0fde59aa8154b09f074e9f214501153eff3277842c478f040d1e513994ac765e"
Jul 13 19:14:27.984298 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:27.984244    2569 scope.go:117] "RemoveContainer" containerID="62357dfad1fd6abb549dd47ca468002d9d425c75924a629fdf16b66f1e770bde"
Jul 13 19:14:27.984481 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:14:27.984456    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-6x8xg_odh-ai-gateway-infra(fe83dc96-c034-4bde-ad90-26c29ad3d10d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d"
Jul 13 19:14:28.020705 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:28.020672    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"]
Jul 13 19:14:29.126188 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:29.126166    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"
Jul 13 19:14:29.210035 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:29.210007    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe83dc96-c034-4bde-ad90-26c29ad3d10d-maas-api-service-ca\") pod \"fe83dc96-c034-4bde-ad90-26c29ad3d10d\" (UID: \"fe83dc96-c034-4bde-ad90-26c29ad3d10d\") "
Jul 13 19:14:29.210399 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:29.210380    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe83dc96-c034-4bde-ad90-26c29ad3d10d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fe83dc96-c034-4bde-ad90-26c29ad3d10d" (UID: "fe83dc96-c034-4bde-ad90-26c29ad3d10d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:14:29.311381 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:29.311279    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe83dc96-c034-4bde-ad90-26c29ad3d10d-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:14:29.997416 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:29.997388    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"
Jul 13 19:14:29.997593 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:29.997430    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg" event={"ID":"fe83dc96-c034-4bde-ad90-26c29ad3d10d","Type":"ContainerDied","Data":"9f05126bfc56e94d09f8b3903513140e71a7b35d2a135eab35abb10c0e72f8a0"}
Jul 13 19:14:29.997593 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:29.997487    2569 scope.go:117] "RemoveContainer" containerID="62357dfad1fd6abb549dd47ca468002d9d425c75924a629fdf16b66f1e770bde"
Jul 13 19:14:30.020338 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:30.020292    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"]
Jul 13 19:14:30.026984 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:30.026959    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-6x8xg"]
Jul 13 19:14:31.350767 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:31.350734    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" path="/var/lib/kubelet/pods/fe83dc96-c034-4bde-ad90-26c29ad3d10d/volumes"
Jul 13 19:14:43.640553 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:43.640509    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:14:58.239327 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:14:58.239290    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:15:00.152763 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.152726    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc"]
Jul 13 19:15:00.153252 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.153233    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerName="revoke-keys"
Jul 13 19:15:00.153341 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.153256    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerName="revoke-keys"
Jul 13 19:15:00.153341 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.153269    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerName="revoke-keys"
Jul 13 19:15:00.153341 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.153277    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerName="revoke-keys"
Jul 13 19:15:00.153490 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.153383    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerName="revoke-keys"
Jul 13 19:15:00.153490 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.153396    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerName="revoke-keys"
Jul 13 19:15:00.157995 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.157973    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x"]
Jul 13 19:15:00.158199 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.158097    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc"
Jul 13 19:15:00.158395 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.158378    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerName="revoke-keys"
Jul 13 19:15:00.158466 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.158398    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerName="revoke-keys"
Jul 13 19:15:00.158521 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.158506    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe83dc96-c034-4bde-ad90-26c29ad3d10d" containerName="revoke-keys"
Jul 13 19:15:00.161197 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.161177    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh"]
Jul 13 19:15:00.161319 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.161276    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x"
Jul 13 19:15:00.162586 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.162568    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:15:00.164357 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.164335    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh"
Jul 13 19:15:00.169810 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.169789    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc"
Jul 13 19:15:00.172001 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.171979    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc"]
Jul 13 19:15:00.179834 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.179811    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh"]
Jul 13 19:15:00.181751 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.181729    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x"
Jul 13 19:15:00.196048 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.195811    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x"]
Jul 13 19:15:00.218959 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.218937    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh"
Jul 13 19:15:00.350620 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.349668    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc"]
Jul 13 19:15:00.350620 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:15:00.350301    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bfd0ca9_48b1_438b_8e35_ece3355a4a45.slice/crio-5261bee4613bed4cddbe8730733affb65631150942501ed7c5c7eced21ffd747 WatchSource:0}: Error finding container 5261bee4613bed4cddbe8730733affb65631150942501ed7c5c7eced21ffd747: Status 404 returned error can't find the container with id 5261bee4613bed4cddbe8730733affb65631150942501ed7c5c7eced21ffd747
Jul 13 19:15:00.378776 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.378588    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x"]
Jul 13 19:15:00.389927 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:15:00.389899    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652d647f_b3d6_455e_b506_ac85dd297026.slice/crio-a4661e01372a15466abb2dc767740443bb0a4a8b39e29ec220d15830ad0d733d WatchSource:0}: Error finding container a4661e01372a15466abb2dc767740443bb0a4a8b39e29ec220d15830ad0d733d: Status 404 returned error can't find the container with id a4661e01372a15466abb2dc767740443bb0a4a8b39e29ec220d15830ad0d733d
Jul 13 19:15:00.414281 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:00.414259    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh"]
Jul 13 19:15:00.425789 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:15:00.425765    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de82b66_f389_4dd4_911d_3d725b9849f4.slice/crio-4722bf585030fca58934a98aed62a1af371e179b3c0364122b2e92b283869ae7 WatchSource:0}: Error finding container 4722bf585030fca58934a98aed62a1af371e179b3c0364122b2e92b283869ae7: Status 404 returned error can't find the container with id 4722bf585030fca58934a98aed62a1af371e179b3c0364122b2e92b283869ae7
Jul 13 19:15:01.179773 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:01.179741    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc" event={"ID":"9bfd0ca9-48b1-438b-8e35-ece3355a4a45","Type":"ContainerStarted","Data":"1da0c111fae37b44d9d60be141f709197e2ce5e0d0973a7d501b587735da7f73"}
Jul 13 19:15:01.180274 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:01.179796    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc" event={"ID":"9bfd0ca9-48b1-438b-8e35-ece3355a4a45","Type":"ContainerStarted","Data":"5261bee4613bed4cddbe8730733affb65631150942501ed7c5c7eced21ffd747"}
Jul 13 19:15:01.181397 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:01.181372    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x" event={"ID":"652d647f-b3d6-455e-b506-ac85dd297026","Type":"ContainerStarted","Data":"ff8196357025be5e17b2ff07e270d9adf9f1c09baec76075048d20130c0522b3"}
Jul 13 19:15:01.181503 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:01.181418    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x" event={"ID":"652d647f-b3d6-455e-b506-ac85dd297026","Type":"ContainerStarted","Data":"a4661e01372a15466abb2dc767740443bb0a4a8b39e29ec220d15830ad0d733d"}
Jul 13 19:15:01.183070 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:01.183047    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh" event={"ID":"5de82b66-f389-4dd4-911d-3d725b9849f4","Type":"ContainerStarted","Data":"a5a399ff4098ca74ec2e04dcc323af559d7fea8f936104178d9711967ac3cd30"}
Jul 13 19:15:01.183156 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:01.183087    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh" event={"ID":"5de82b66-f389-4dd4-911d-3d725b9849f4","Type":"ContainerStarted","Data":"4722bf585030fca58934a98aed62a1af371e179b3c0364122b2e92b283869ae7"}
Jul 13 19:15:01.199170 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:01.199112    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc" podStartSLOduration=1.199095405 podStartE2EDuration="1.199095405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:15:01.196421957 +0000 UTC m=+2582.327710416" watchObservedRunningTime="2026-07-13 19:15:01.199095405 +0000 UTC m=+2582.330383866"
Jul 13 19:15:01.219939 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:01.219895    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x" podStartSLOduration=1.21988065 podStartE2EDuration="1.21988065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:15:01.216570182 +0000 UTC m=+2582.347858642" watchObservedRunningTime="2026-07-13 19:15:01.21988065 +0000 UTC m=+2582.351169109"
Jul 13 19:15:01.241129 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:01.241082    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh" podStartSLOduration=1.241068876 podStartE2EDuration="1.241068876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:15:01.238966459 +0000 UTC m=+2582.370254922" watchObservedRunningTime="2026-07-13 19:15:01.241068876 +0000 UTC m=+2582.372357365"
Jul 13 19:15:02.190277 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.190238    2569 generic.go:356] "Generic (PLEG): container finished" podID="9bfd0ca9-48b1-438b-8e35-ece3355a4a45" containerID="1da0c111fae37b44d9d60be141f709197e2ce5e0d0973a7d501b587735da7f73" exitCode=0
Jul 13 19:15:02.190728 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.190323    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc" event={"ID":"9bfd0ca9-48b1-438b-8e35-ece3355a4a45","Type":"ContainerDied","Data":"1da0c111fae37b44d9d60be141f709197e2ce5e0d0973a7d501b587735da7f73"}
Jul 13 19:15:02.191814 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.191791    2569 generic.go:356] "Generic (PLEG): container finished" podID="652d647f-b3d6-455e-b506-ac85dd297026" containerID="ff8196357025be5e17b2ff07e270d9adf9f1c09baec76075048d20130c0522b3" exitCode=0
Jul 13 19:15:02.191903 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.191876    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x" event={"ID":"652d647f-b3d6-455e-b506-ac85dd297026","Type":"ContainerDied","Data":"ff8196357025be5e17b2ff07e270d9adf9f1c09baec76075048d20130c0522b3"}
Jul 13 19:15:02.193197 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.193168    2569 generic.go:356] "Generic (PLEG): container finished" podID="5de82b66-f389-4dd4-911d-3d725b9849f4" containerID="a5a399ff4098ca74ec2e04dcc323af559d7fea8f936104178d9711967ac3cd30" exitCode=0
Jul 13 19:15:02.193273 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.193199    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh" event={"ID":"5de82b66-f389-4dd4-911d-3d725b9849f4","Type":"ContainerDied","Data":"a5a399ff4098ca74ec2e04dcc323af559d7fea8f936104178d9711967ac3cd30"}
Jul 13 19:15:02.341024 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.340987    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"]
Jul 13 19:15:02.344511 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.344489    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"
Jul 13 19:15:02.353909 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.353875    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"]
Jul 13 19:15:02.403883 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.403842    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8d4b5f-1114-416a-b9ef-3b7e26625c4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-99xhs\" (UID: \"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"
Jul 13 19:15:02.504795 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.504756    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8d4b5f-1114-416a-b9ef-3b7e26625c4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-99xhs\" (UID: \"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"
Jul 13 19:15:02.505400 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.505380    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8d4b5f-1114-416a-b9ef-3b7e26625c4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-99xhs\" (UID: \"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"
Jul 13 19:15:02.656523 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.656485    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"
Jul 13 19:15:02.791083 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:02.791057    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"]
Jul 13 19:15:02.792672 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:15:02.792627    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a8d4b5f_1114_416a_b9ef_3b7e26625c4c.slice/crio-07a579b9b375b325b7663302b135c70e48b5b58b4f99441fb6f91a974bfb9889 WatchSource:0}: Error finding container 07a579b9b375b325b7663302b135c70e48b5b58b4f99441fb6f91a974bfb9889: Status 404 returned error can't find the container with id 07a579b9b375b325b7663302b135c70e48b5b58b4f99441fb6f91a974bfb9889
Jul 13 19:15:03.202853 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:03.202762    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs" event={"ID":"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c","Type":"ContainerStarted","Data":"0eec0f1c1e33ec186a2b9ccf4cd60af710813552ede4b88da9295e79931cfaed"}
Jul 13 19:15:03.202853 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:03.202820    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs" event={"ID":"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c","Type":"ContainerStarted","Data":"07a579b9b375b325b7663302b135c70e48b5b58b4f99441fb6f91a974bfb9889"}
Jul 13 19:15:03.222671 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:03.222592    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs" podStartSLOduration=1.222569751 podStartE2EDuration="1.222569751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:15:03.219672192 +0000 UTC m=+2584.350960653" watchObservedRunningTime="2026-07-13 19:15:03.222569751 +0000 UTC m=+2584.353858213"
Jul 13 19:15:03.352734 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:03.352710    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x"
Jul 13 19:15:03.379748 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:03.379722    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc"
Jul 13 19:15:03.393634 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:03.393612    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh"
Jul 13 19:15:04.209799 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.209769    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh"
Jul 13 19:15:04.209799 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.209788    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732835-nl7hh" event={"ID":"5de82b66-f389-4dd4-911d-3d725b9849f4","Type":"ContainerDied","Data":"4722bf585030fca58934a98aed62a1af371e179b3c0364122b2e92b283869ae7"}
Jul 13 19:15:04.210196 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.209830    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4722bf585030fca58934a98aed62a1af371e179b3c0364122b2e92b283869ae7"
Jul 13 19:15:04.211379 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.211351    2569 generic.go:356] "Generic (PLEG): container finished" podID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerID="0eec0f1c1e33ec186a2b9ccf4cd60af710813552ede4b88da9295e79931cfaed" exitCode=6
Jul 13 19:15:04.211521 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.211444    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs" event={"ID":"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c","Type":"ContainerDied","Data":"0eec0f1c1e33ec186a2b9ccf4cd60af710813552ede4b88da9295e79931cfaed"}
Jul 13 19:15:04.211619 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.211603    2569 scope.go:117] "RemoveContainer" containerID="0eec0f1c1e33ec186a2b9ccf4cd60af710813552ede4b88da9295e79931cfaed"
Jul 13 19:15:04.212837 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.212806    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc" event={"ID":"9bfd0ca9-48b1-438b-8e35-ece3355a4a45","Type":"ContainerDied","Data":"5261bee4613bed4cddbe8730733affb65631150942501ed7c5c7eced21ffd747"}
Jul 13 19:15:04.212931 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.212845    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5261bee4613bed4cddbe8730733affb65631150942501ed7c5c7eced21ffd747"
Jul 13 19:15:04.212931 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.212814    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc"
Jul 13 19:15:04.214172 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.214139    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x" event={"ID":"652d647f-b3d6-455e-b506-ac85dd297026","Type":"ContainerDied","Data":"a4661e01372a15466abb2dc767740443bb0a4a8b39e29ec220d15830ad0d733d"}
Jul 13 19:15:04.214172 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.214161    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x"
Jul 13 19:15:04.214341 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:04.214183    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4661e01372a15466abb2dc767740443bb0a4a8b39e29ec220d15830ad0d733d"
Jul 13 19:15:05.226215 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:05.226181    2569 generic.go:356] "Generic (PLEG): container finished" podID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerID="d687a158b1d5dc2244d426ac6af627cb106ae18df9fadb6dbb86848e72bd66ba" exitCode=6
Jul 13 19:15:05.226714 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:05.226233    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs" event={"ID":"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c","Type":"ContainerDied","Data":"d687a158b1d5dc2244d426ac6af627cb106ae18df9fadb6dbb86848e72bd66ba"}
Jul 13 19:15:05.226714 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:05.226308    2569 scope.go:117] "RemoveContainer" containerID="0eec0f1c1e33ec186a2b9ccf4cd60af710813552ede4b88da9295e79931cfaed"
Jul 13 19:15:05.226714 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:05.226530    2569 scope.go:117] "RemoveContainer" containerID="d687a158b1d5dc2244d426ac6af627cb106ae18df9fadb6dbb86848e72bd66ba"
Jul 13 19:15:05.226884 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:15:05.226800    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-99xhs_odh-ai-gateway-infra(1a8d4b5f-1114-416a-b9ef-3b7e26625c4c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c"
Jul 13 19:15:06.233177 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:06.233150    2569 scope.go:117] "RemoveContainer" containerID="d687a158b1d5dc2244d426ac6af627cb106ae18df9fadb6dbb86848e72bd66ba"
Jul 13 19:15:06.233540 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:15:06.233337    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-99xhs_odh-ai-gateway-infra(1a8d4b5f-1114-416a-b9ef-3b7e26625c4c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c"
Jul 13 19:15:15.743365 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:15.743321    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:15:21.344604 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:21.344515    2569 scope.go:117] "RemoveContainer" containerID="d687a158b1d5dc2244d426ac6af627cb106ae18df9fadb6dbb86848e72bd66ba"
Jul 13 19:15:22.323396 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:22.323364    2569 generic.go:356] "Generic (PLEG): container finished" podID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerID="1969a7fe03e2df363b7c19318fc862223e45a919f80464d42822d60b5b40a35a" exitCode=6
Jul 13 19:15:22.323810 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:22.323438    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs" event={"ID":"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c","Type":"ContainerDied","Data":"1969a7fe03e2df363b7c19318fc862223e45a919f80464d42822d60b5b40a35a"}
Jul 13 19:15:22.323810 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:22.323502    2569 scope.go:117] "RemoveContainer" containerID="d687a158b1d5dc2244d426ac6af627cb106ae18df9fadb6dbb86848e72bd66ba"
Jul 13 19:15:22.323967 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:22.323823    2569 scope.go:117] "RemoveContainer" containerID="1969a7fe03e2df363b7c19318fc862223e45a919f80464d42822d60b5b40a35a"
Jul 13 19:15:22.324040 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:15:22.324018    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-99xhs_odh-ai-gateway-infra(1a8d4b5f-1114-416a-b9ef-3b7e26625c4c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c"
Jul 13 19:15:23.350495 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:23.350466    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"]
Jul 13 19:15:23.492096 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:23.492072    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"
Jul 13 19:15:23.596621 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:23.596585    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8d4b5f-1114-416a-b9ef-3b7e26625c4c-maas-api-service-ca\") pod \"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c\" (UID: \"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c\") "
Jul 13 19:15:23.596978 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:23.596956    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a8d4b5f-1114-416a-b9ef-3b7e26625c4c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" (UID: "1a8d4b5f-1114-416a-b9ef-3b7e26625c4c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:15:23.698075 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:23.698001    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8d4b5f-1114-416a-b9ef-3b7e26625c4c-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:15:24.338015 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:24.337985    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"
Jul 13 19:15:24.338015 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:24.338001    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs" event={"ID":"1a8d4b5f-1114-416a-b9ef-3b7e26625c4c","Type":"ContainerDied","Data":"07a579b9b375b325b7663302b135c70e48b5b58b4f99441fb6f91a974bfb9889"}
Jul 13 19:15:24.338273 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:24.338084    2569 scope.go:117] "RemoveContainer" containerID="1969a7fe03e2df363b7c19318fc862223e45a919f80464d42822d60b5b40a35a"
Jul 13 19:15:24.361448 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:24.361423    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"]
Jul 13 19:15:24.368978 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:24.368955    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-99xhs"]
Jul 13 19:15:25.349375 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:25.349344    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" path="/var/lib/kubelet/pods/1a8d4b5f-1114-416a-b9ef-3b7e26625c4c/volumes"
Jul 13 19:15:33.040348 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:33.040314    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:15:51.944688 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:51.944640    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:15:59.632777 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:15:59.632741    2569 scope.go:117] "RemoveContainer" containerID="c79c7b26dd635cc022e278bf99bbe7764d47a46c8902b677bc2d498b98506c4c"
Jul 13 19:16:02.400707 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.400669    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"]
Jul 13 19:16:02.401121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401053    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerName="revoke-keys"
Jul 13 19:16:02.401121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401066    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerName="revoke-keys"
Jul 13 19:16:02.401121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401076    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bfd0ca9-48b1-438b-8e35-ece3355a4a45" containerName="cleanup"
Jul 13 19:16:02.401121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401082    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bfd0ca9-48b1-438b-8e35-ece3355a4a45" containerName="cleanup"
Jul 13 19:16:02.401121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401089    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="652d647f-b3d6-455e-b506-ac85dd297026" containerName="cleanup"
Jul 13 19:16:02.401121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401095    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="652d647f-b3d6-455e-b506-ac85dd297026" containerName="cleanup"
Jul 13 19:16:02.401121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401108    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5de82b66-f389-4dd4-911d-3d725b9849f4" containerName="cleanup"
Jul 13 19:16:02.401121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401119    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5de82b66-f389-4dd4-911d-3d725b9849f4" containerName="cleanup"
Jul 13 19:16:02.401121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401126    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerName="revoke-keys"
Jul 13 19:16:02.401409 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401132    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerName="revoke-keys"
Jul 13 19:16:02.401409 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401194    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerName="revoke-keys"
Jul 13 19:16:02.401409 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401201    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="652d647f-b3d6-455e-b506-ac85dd297026" containerName="cleanup"
Jul 13 19:16:02.401409 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401209    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bfd0ca9-48b1-438b-8e35-ece3355a4a45" containerName="cleanup"
Jul 13 19:16:02.401409 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401214    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5de82b66-f389-4dd4-911d-3d725b9849f4" containerName="cleanup"
Jul 13 19:16:02.401409 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.401220    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerName="revoke-keys"
Jul 13 19:16:02.405914 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.405895    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"
Jul 13 19:16:02.408617 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.408596    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:16:02.414059 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.414017    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"]
Jul 13 19:16:02.546282 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.546249    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71086f30-c6e6-4972-877b-ee817bdd274e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-s8rcd\" (UID: \"71086f30-c6e6-4972-877b-ee817bdd274e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"
Jul 13 19:16:02.647366 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.647325    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71086f30-c6e6-4972-877b-ee817bdd274e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-s8rcd\" (UID: \"71086f30-c6e6-4972-877b-ee817bdd274e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"
Jul 13 19:16:02.648017 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.647987    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71086f30-c6e6-4972-877b-ee817bdd274e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-s8rcd\" (UID: \"71086f30-c6e6-4972-877b-ee817bdd274e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"
Jul 13 19:16:02.725510 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.725478    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"
Jul 13 19:16:02.863582 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:02.863556    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"]
Jul 13 19:16:02.865193 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:16:02.865152    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71086f30_c6e6_4972_877b_ee817bdd274e.slice/crio-bfdf31e7eb9ce0b0a1c7ca951d4b1f97a161ab5432ff0ce04dc2c0310bd88fdb WatchSource:0}: Error finding container bfdf31e7eb9ce0b0a1c7ca951d4b1f97a161ab5432ff0ce04dc2c0310bd88fdb: Status 404 returned error can't find the container with id bfdf31e7eb9ce0b0a1c7ca951d4b1f97a161ab5432ff0ce04dc2c0310bd88fdb
Jul 13 19:16:03.554192 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:03.554155    2569 generic.go:356] "Generic (PLEG): container finished" podID="71086f30-c6e6-4972-877b-ee817bdd274e" containerID="f4e901e8a72a44ebf81a770fe1164753c098c1ead3c9dd79c79321fff39e7c47" exitCode=6
Jul 13 19:16:03.554588 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:03.554243    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd" event={"ID":"71086f30-c6e6-4972-877b-ee817bdd274e","Type":"ContainerDied","Data":"f4e901e8a72a44ebf81a770fe1164753c098c1ead3c9dd79c79321fff39e7c47"}
Jul 13 19:16:03.554588 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:03.554315    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd" event={"ID":"71086f30-c6e6-4972-877b-ee817bdd274e","Type":"ContainerStarted","Data":"bfdf31e7eb9ce0b0a1c7ca951d4b1f97a161ab5432ff0ce04dc2c0310bd88fdb"}
Jul 13 19:16:03.554588 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:03.554391    2569 scope.go:117] "RemoveContainer" containerID="f4e901e8a72a44ebf81a770fe1164753c098c1ead3c9dd79c79321fff39e7c47"
Jul 13 19:16:04.562317 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:04.562284    2569 generic.go:356] "Generic (PLEG): container finished" podID="71086f30-c6e6-4972-877b-ee817bdd274e" containerID="fda586811fa63c8d46fa664f6889f8270befd47ad94de65a71af0580907b117b" exitCode=6
Jul 13 19:16:04.562805 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:04.562334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd" event={"ID":"71086f30-c6e6-4972-877b-ee817bdd274e","Type":"ContainerDied","Data":"fda586811fa63c8d46fa664f6889f8270befd47ad94de65a71af0580907b117b"}
Jul 13 19:16:04.562805 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:04.562387    2569 scope.go:117] "RemoveContainer" containerID="f4e901e8a72a44ebf81a770fe1164753c098c1ead3c9dd79c79321fff39e7c47"
Jul 13 19:16:04.562805 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:04.562548    2569 scope.go:117] "RemoveContainer" containerID="fda586811fa63c8d46fa664f6889f8270befd47ad94de65a71af0580907b117b"
Jul 13 19:16:04.562938 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:16:04.562804    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-s8rcd_odh-ai-gateway-infra(71086f30-c6e6-4972-877b-ee817bdd274e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd" podUID="71086f30-c6e6-4972-877b-ee817bdd274e"
Jul 13 19:16:05.570393 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:05.570369    2569 scope.go:117] "RemoveContainer" containerID="fda586811fa63c8d46fa664f6889f8270befd47ad94de65a71af0580907b117b"
Jul 13 19:16:05.570797 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:16:05.570544    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-s8rcd_odh-ai-gateway-infra(71086f30-c6e6-4972-877b-ee817bdd274e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd" podUID="71086f30-c6e6-4972-877b-ee817bdd274e"
Jul 13 19:16:10.136432 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:10.136399    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:16:18.345083 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:18.345043    2569 scope.go:117] "RemoveContainer" containerID="fda586811fa63c8d46fa664f6889f8270befd47ad94de65a71af0580907b117b"
Jul 13 19:16:18.647332 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:18.647303    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd" event={"ID":"71086f30-c6e6-4972-877b-ee817bdd274e","Type":"ContainerStarted","Data":"d79e06e52dcfaa36581c1c9f9080bb40f862ded8250ca3538de7708f38a5f9b7"}
Jul 13 19:16:18.665574 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:18.665527    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd" podStartSLOduration=16.665512214 podStartE2EDuration="16.665512214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:16:18.663517826 +0000 UTC m=+2659.794806297" watchObservedRunningTime="2026-07-13 19:16:18.665512214 +0000 UTC m=+2659.796800673"
Jul 13 19:16:19.655259 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:19.655226    2569 generic.go:356] "Generic (PLEG): container finished" podID="71086f30-c6e6-4972-877b-ee817bdd274e" containerID="d79e06e52dcfaa36581c1c9f9080bb40f862ded8250ca3538de7708f38a5f9b7" exitCode=6
Jul 13 19:16:19.655765 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:19.655297    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd" event={"ID":"71086f30-c6e6-4972-877b-ee817bdd274e","Type":"ContainerDied","Data":"d79e06e52dcfaa36581c1c9f9080bb40f862ded8250ca3538de7708f38a5f9b7"}
Jul 13 19:16:19.655765 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:19.655360    2569 scope.go:117] "RemoveContainer" containerID="fda586811fa63c8d46fa664f6889f8270befd47ad94de65a71af0580907b117b"
Jul 13 19:16:19.655765 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:19.655689    2569 scope.go:117] "RemoveContainer" containerID="d79e06e52dcfaa36581c1c9f9080bb40f862ded8250ca3538de7708f38a5f9b7"
Jul 13 19:16:19.655973 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:16:19.655953    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-s8rcd_odh-ai-gateway-infra(71086f30-c6e6-4972-877b-ee817bdd274e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd" podUID="71086f30-c6e6-4972-877b-ee817bdd274e"
Jul 13 19:16:19.673551 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:19.673526    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"]
Jul 13 19:16:20.796157 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:20.796130    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"
Jul 13 19:16:20.807706 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:20.807676    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71086f30-c6e6-4972-877b-ee817bdd274e-maas-api-service-ca\") pod \"71086f30-c6e6-4972-877b-ee817bdd274e\" (UID: \"71086f30-c6e6-4972-877b-ee817bdd274e\") "
Jul 13 19:16:20.807980 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:20.807958    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71086f30-c6e6-4972-877b-ee817bdd274e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71086f30-c6e6-4972-877b-ee817bdd274e" (UID: "71086f30-c6e6-4972-877b-ee817bdd274e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:16:20.909033 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:20.908989    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71086f30-c6e6-4972-877b-ee817bdd274e-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:16:21.669578 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:21.669489    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"
Jul 13 19:16:21.669578 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:21.669520    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd" event={"ID":"71086f30-c6e6-4972-877b-ee817bdd274e","Type":"ContainerDied","Data":"bfdf31e7eb9ce0b0a1c7ca951d4b1f97a161ab5432ff0ce04dc2c0310bd88fdb"}
Jul 13 19:16:21.669861 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:21.669596    2569 scope.go:117] "RemoveContainer" containerID="d79e06e52dcfaa36581c1c9f9080bb40f862ded8250ca3538de7708f38a5f9b7"
Jul 13 19:16:21.686000 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:21.685978    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"]
Jul 13 19:16:21.692964 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:21.692942    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-s8rcd"]
Jul 13 19:16:23.351469 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:23.351434    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71086f30-c6e6-4972-877b-ee817bdd274e" path="/var/lib/kubelet/pods/71086f30-c6e6-4972-877b-ee817bdd274e/volumes"
Jul 13 19:16:27.837110 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:27.837072    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:16:50.047808 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:50.047728    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:16:52.441246 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441205    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"]
Jul 13 19:16:52.441750 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441730    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71086f30-c6e6-4972-877b-ee817bdd274e" containerName="revoke-keys"
Jul 13 19:16:52.441750 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441749    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="71086f30-c6e6-4972-877b-ee817bdd274e" containerName="revoke-keys"
Jul 13 19:16:52.441890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441773    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerName="revoke-keys"
Jul 13 19:16:52.441890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441781    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerName="revoke-keys"
Jul 13 19:16:52.441890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441827    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71086f30-c6e6-4972-877b-ee817bdd274e" containerName="revoke-keys"
Jul 13 19:16:52.441890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441837    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="71086f30-c6e6-4972-877b-ee817bdd274e" containerName="revoke-keys"
Jul 13 19:16:52.442089 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441934    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="71086f30-c6e6-4972-877b-ee817bdd274e" containerName="revoke-keys"
Jul 13 19:16:52.442089 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441947    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="71086f30-c6e6-4972-877b-ee817bdd274e" containerName="revoke-keys"
Jul 13 19:16:52.442089 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441957    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8d4b5f-1114-416a-b9ef-3b7e26625c4c" containerName="revoke-keys"
Jul 13 19:16:52.442089 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.441967    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="71086f30-c6e6-4972-877b-ee817bdd274e" containerName="revoke-keys"
Jul 13 19:16:52.445204 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.445184    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"
Jul 13 19:16:52.447809 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.447791    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:16:52.452319 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.452188    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"]
Jul 13 19:16:52.595123 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.595091    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bbf514c-9165-4cff-9702-214e661b2237-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-dj52j\" (UID: \"1bbf514c-9165-4cff-9702-214e661b2237\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"
Jul 13 19:16:52.696440 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.696338    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bbf514c-9165-4cff-9702-214e661b2237-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-dj52j\" (UID: \"1bbf514c-9165-4cff-9702-214e661b2237\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"
Jul 13 19:16:52.697045 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.697018    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bbf514c-9165-4cff-9702-214e661b2237-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-dj52j\" (UID: \"1bbf514c-9165-4cff-9702-214e661b2237\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"
Jul 13 19:16:52.757393 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.757360    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"
Jul 13 19:16:52.899571 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:52.899540    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"]
Jul 13 19:16:52.901239 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:16:52.901211    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bbf514c_9165_4cff_9702_214e661b2237.slice/crio-012689aaf3f45c2996d54c8e4fbe6df1e2c4af7994580bbc32063ff3fe62ac04 WatchSource:0}: Error finding container 012689aaf3f45c2996d54c8e4fbe6df1e2c4af7994580bbc32063ff3fe62ac04: Status 404 returned error can't find the container with id 012689aaf3f45c2996d54c8e4fbe6df1e2c4af7994580bbc32063ff3fe62ac04
Jul 13 19:16:53.852428 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:53.852335    2569 generic.go:356] "Generic (PLEG): container finished" podID="1bbf514c-9165-4cff-9702-214e661b2237" containerID="0fb737096bb17a4e58824495c549226a73f4bfa87608c7f83a859873d511114f" exitCode=6
Jul 13 19:16:53.852428 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:53.852379    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j" event={"ID":"1bbf514c-9165-4cff-9702-214e661b2237","Type":"ContainerDied","Data":"0fb737096bb17a4e58824495c549226a73f4bfa87608c7f83a859873d511114f"}
Jul 13 19:16:53.852428 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:53.852418    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j" event={"ID":"1bbf514c-9165-4cff-9702-214e661b2237","Type":"ContainerStarted","Data":"012689aaf3f45c2996d54c8e4fbe6df1e2c4af7994580bbc32063ff3fe62ac04"}
Jul 13 19:16:53.852940 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:53.852614    2569 scope.go:117] "RemoveContainer" containerID="0fb737096bb17a4e58824495c549226a73f4bfa87608c7f83a859873d511114f"
Jul 13 19:16:54.860056 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:54.859964    2569 generic.go:356] "Generic (PLEG): container finished" podID="1bbf514c-9165-4cff-9702-214e661b2237" containerID="61dcffc750616f4c1f0613f3048c621989f22a05e24e696220349861c260a7b0" exitCode=6
Jul 13 19:16:54.860520 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:54.860051    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j" event={"ID":"1bbf514c-9165-4cff-9702-214e661b2237","Type":"ContainerDied","Data":"61dcffc750616f4c1f0613f3048c621989f22a05e24e696220349861c260a7b0"}
Jul 13 19:16:54.860520 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:54.860107    2569 scope.go:117] "RemoveContainer" containerID="0fb737096bb17a4e58824495c549226a73f4bfa87608c7f83a859873d511114f"
Jul 13 19:16:54.860520 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:54.860327    2569 scope.go:117] "RemoveContainer" containerID="61dcffc750616f4c1f0613f3048c621989f22a05e24e696220349861c260a7b0"
Jul 13 19:16:54.860684 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:16:54.860548    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-dj52j_odh-ai-gateway-infra(1bbf514c-9165-4cff-9702-214e661b2237)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j" podUID="1bbf514c-9165-4cff-9702-214e661b2237"
Jul 13 19:16:55.867406 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:55.867376    2569 scope.go:117] "RemoveContainer" containerID="61dcffc750616f4c1f0613f3048c621989f22a05e24e696220349861c260a7b0"
Jul 13 19:16:55.868058 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:16:55.867597    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-dj52j_odh-ai-gateway-infra(1bbf514c-9165-4cff-9702-214e661b2237)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j" podUID="1bbf514c-9165-4cff-9702-214e661b2237"
Jul 13 19:16:59.504642 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:59.504616    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 19:16:59.512536 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:16:59.512513    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r6wmn_10a8e7a5-f7f3-4b20-847a-f8563ff427c7/ovn-acl-logging/0.log"
Jul 13 19:17:04.241586 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:04.241552    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:17:09.348400 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:09.348371    2569 scope.go:117] "RemoveContainer" containerID="61dcffc750616f4c1f0613f3048c621989f22a05e24e696220349861c260a7b0"
Jul 13 19:17:09.948479 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:09.948448    2569 generic.go:356] "Generic (PLEG): container finished" podID="1bbf514c-9165-4cff-9702-214e661b2237" containerID="d6a415439eb485e324f33efc91129999f6eeb0b518bd614557eaa6b21904f56d" exitCode=6
Jul 13 19:17:09.948615 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:09.948521    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j" event={"ID":"1bbf514c-9165-4cff-9702-214e661b2237","Type":"ContainerDied","Data":"d6a415439eb485e324f33efc91129999f6eeb0b518bd614557eaa6b21904f56d"}
Jul 13 19:17:09.948615 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:09.948583    2569 scope.go:117] "RemoveContainer" containerID="61dcffc750616f4c1f0613f3048c621989f22a05e24e696220349861c260a7b0"
Jul 13 19:17:09.948902 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:09.948882    2569 scope.go:117] "RemoveContainer" containerID="d6a415439eb485e324f33efc91129999f6eeb0b518bd614557eaa6b21904f56d"
Jul 13 19:17:09.949116 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:17:09.949092    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-dj52j_odh-ai-gateway-infra(1bbf514c-9165-4cff-9702-214e661b2237)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j" podUID="1bbf514c-9165-4cff-9702-214e661b2237"
Jul 13 19:17:10.981772 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:10.981739    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"]
Jul 13 19:17:11.124580 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:11.124556    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"
Jul 13 19:17:11.155582 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:11.155557    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bbf514c-9165-4cff-9702-214e661b2237-maas-api-service-ca\") pod \"1bbf514c-9165-4cff-9702-214e661b2237\" (UID: \"1bbf514c-9165-4cff-9702-214e661b2237\") "
Jul 13 19:17:11.155906 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:11.155884    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bbf514c-9165-4cff-9702-214e661b2237-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1bbf514c-9165-4cff-9702-214e661b2237" (UID: "1bbf514c-9165-4cff-9702-214e661b2237"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:17:11.256270 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:11.256188    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bbf514c-9165-4cff-9702-214e661b2237-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:17:11.962466 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:11.962440    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"
Jul 13 19:17:11.962613 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:11.962435    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j" event={"ID":"1bbf514c-9165-4cff-9702-214e661b2237","Type":"ContainerDied","Data":"012689aaf3f45c2996d54c8e4fbe6df1e2c4af7994580bbc32063ff3fe62ac04"}
Jul 13 19:17:11.962613 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:11.962582    2569 scope.go:117] "RemoveContainer" containerID="d6a415439eb485e324f33efc91129999f6eeb0b518bd614557eaa6b21904f56d"
Jul 13 19:17:11.983202 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:11.983174    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"]
Jul 13 19:17:11.988161 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:11.988133    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-dj52j"]
Jul 13 19:17:13.350424 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:13.350387    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bbf514c-9165-4cff-9702-214e661b2237" path="/var/lib/kubelet/pods/1bbf514c-9165-4cff-9702-214e661b2237/volumes"
Jul 13 19:17:24.138716 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:24.138679    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:17:40.334996 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:40.334956    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:17:42.494899 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.494857    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"]
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495216    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bbf514c-9165-4cff-9702-214e661b2237" containerName="revoke-keys"
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495226    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bbf514c-9165-4cff-9702-214e661b2237" containerName="revoke-keys"
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495238    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bbf514c-9165-4cff-9702-214e661b2237" containerName="revoke-keys"
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495243    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bbf514c-9165-4cff-9702-214e661b2237" containerName="revoke-keys"
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495255    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bbf514c-9165-4cff-9702-214e661b2237" containerName="revoke-keys"
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495260    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bbf514c-9165-4cff-9702-214e661b2237" containerName="revoke-keys"
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495272    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71086f30-c6e6-4972-877b-ee817bdd274e" containerName="revoke-keys"
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495277    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="71086f30-c6e6-4972-877b-ee817bdd274e" containerName="revoke-keys"
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495326    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bbf514c-9165-4cff-9702-214e661b2237" containerName="revoke-keys"
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495334    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bbf514c-9165-4cff-9702-214e661b2237" containerName="revoke-keys"
Jul 13 19:17:42.495376 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.495341    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bbf514c-9165-4cff-9702-214e661b2237" containerName="revoke-keys"
Jul 13 19:17:42.498096 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.498077    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"
Jul 13 19:17:42.501154 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.501126    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:17:42.508809 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.508787    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"]
Jul 13 19:17:42.623848 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.623811    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/917138d3-6997-41e2-b523-80cceb9d1c5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-kpd2m\" (UID: \"917138d3-6997-41e2-b523-80cceb9d1c5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"
Jul 13 19:17:42.724237 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.724207    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/917138d3-6997-41e2-b523-80cceb9d1c5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-kpd2m\" (UID: \"917138d3-6997-41e2-b523-80cceb9d1c5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"
Jul 13 19:17:42.724903 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.724878    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/917138d3-6997-41e2-b523-80cceb9d1c5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-kpd2m\" (UID: \"917138d3-6997-41e2-b523-80cceb9d1c5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"
Jul 13 19:17:42.810211 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.810115    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"
Jul 13 19:17:42.944706 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:42.944684    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"]
Jul 13 19:17:42.946559 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:17:42.946529    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod917138d3_6997_41e2_b523_80cceb9d1c5c.slice/crio-9b17509504ce03373d3495e541c9b5815cb2a2b64c6ef8b7e7515654685f2311 WatchSource:0}: Error finding container 9b17509504ce03373d3495e541c9b5815cb2a2b64c6ef8b7e7515654685f2311: Status 404 returned error can't find the container with id 9b17509504ce03373d3495e541c9b5815cb2a2b64c6ef8b7e7515654685f2311
Jul 13 19:17:43.142969 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:43.142931    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m" event={"ID":"917138d3-6997-41e2-b523-80cceb9d1c5c","Type":"ContainerStarted","Data":"9b17509504ce03373d3495e541c9b5815cb2a2b64c6ef8b7e7515654685f2311"}
Jul 13 19:17:44.150531 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:44.150496    2569 generic.go:356] "Generic (PLEG): container finished" podID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerID="dd912c1f0d20089402711be9e1bff92797fbc5eee34d7ed0a2af1465ba0d3afa" exitCode=6
Jul 13 19:17:44.150934 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:44.150548    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m" event={"ID":"917138d3-6997-41e2-b523-80cceb9d1c5c","Type":"ContainerDied","Data":"dd912c1f0d20089402711be9e1bff92797fbc5eee34d7ed0a2af1465ba0d3afa"}
Jul 13 19:17:44.150934 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:44.150773    2569 scope.go:117] "RemoveContainer" containerID="dd912c1f0d20089402711be9e1bff92797fbc5eee34d7ed0a2af1465ba0d3afa"
Jul 13 19:17:45.158214 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:45.158172    2569 generic.go:356] "Generic (PLEG): container finished" podID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerID="db4618933baf696c95db18634e42380fd803c661acd9680600eec871bfcc5b20" exitCode=6
Jul 13 19:17:45.158674 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:45.158218    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m" event={"ID":"917138d3-6997-41e2-b523-80cceb9d1c5c","Type":"ContainerDied","Data":"db4618933baf696c95db18634e42380fd803c661acd9680600eec871bfcc5b20"}
Jul 13 19:17:45.158674 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:45.158291    2569 scope.go:117] "RemoveContainer" containerID="dd912c1f0d20089402711be9e1bff92797fbc5eee34d7ed0a2af1465ba0d3afa"
Jul 13 19:17:45.158674 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:45.158487    2569 scope.go:117] "RemoveContainer" containerID="db4618933baf696c95db18634e42380fd803c661acd9680600eec871bfcc5b20"
Jul 13 19:17:45.158839 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:17:45.158734    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-kpd2m_odh-ai-gateway-infra(917138d3-6997-41e2-b523-80cceb9d1c5c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c"
Jul 13 19:17:46.165856 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:46.165829    2569 scope.go:117] "RemoveContainer" containerID="db4618933baf696c95db18634e42380fd803c661acd9680600eec871bfcc5b20"
Jul 13 19:17:46.166351 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:17:46.166074    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-kpd2m_odh-ai-gateway-infra(917138d3-6997-41e2-b523-80cceb9d1c5c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c"
Jul 13 19:17:59.669348 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:17:59.669317    2569 scope.go:117] "RemoveContainer" containerID="4d65db44428bfcc2649749ec344a56a36c580fde6fb1c8ef59374cc11252da39"
Jul 13 19:18:00.344459 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:00.344423    2569 scope.go:117] "RemoveContainer" containerID="db4618933baf696c95db18634e42380fd803c661acd9680600eec871bfcc5b20"
Jul 13 19:18:01.251581 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:01.251548    2569 generic.go:356] "Generic (PLEG): container finished" podID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerID="36377bd97a1ad6e2ad629d5a05774fba7b410fc2c1edbc2bb68e4be07f84997c" exitCode=6
Jul 13 19:18:01.252086 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:01.251603    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m" event={"ID":"917138d3-6997-41e2-b523-80cceb9d1c5c","Type":"ContainerDied","Data":"36377bd97a1ad6e2ad629d5a05774fba7b410fc2c1edbc2bb68e4be07f84997c"}
Jul 13 19:18:01.252086 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:01.251677    2569 scope.go:117] "RemoveContainer" containerID="db4618933baf696c95db18634e42380fd803c661acd9680600eec871bfcc5b20"
Jul 13 19:18:01.252086 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:01.252027    2569 scope.go:117] "RemoveContainer" containerID="36377bd97a1ad6e2ad629d5a05774fba7b410fc2c1edbc2bb68e4be07f84997c"
Jul 13 19:18:01.252288 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:18:01.252264    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-kpd2m_odh-ai-gateway-infra(917138d3-6997-41e2-b523-80cceb9d1c5c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c"
Jul 13 19:18:02.280292 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:02.280258    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"]
Jul 13 19:18:02.418133 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:02.418111    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"
Jul 13 19:18:02.510354 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:02.510318    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/917138d3-6997-41e2-b523-80cceb9d1c5c-maas-api-service-ca\") pod \"917138d3-6997-41e2-b523-80cceb9d1c5c\" (UID: \"917138d3-6997-41e2-b523-80cceb9d1c5c\") "
Jul 13 19:18:02.510734 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:02.510709    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/917138d3-6997-41e2-b523-80cceb9d1c5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "917138d3-6997-41e2-b523-80cceb9d1c5c" (UID: "917138d3-6997-41e2-b523-80cceb9d1c5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:18:02.611417 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:02.611338    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/917138d3-6997-41e2-b523-80cceb9d1c5c-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:18:02.739351 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:02.739317    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:18:03.266175 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:03.266147    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"
Jul 13 19:18:03.266349 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:03.266188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m" event={"ID":"917138d3-6997-41e2-b523-80cceb9d1c5c","Type":"ContainerDied","Data":"9b17509504ce03373d3495e541c9b5815cb2a2b64c6ef8b7e7515654685f2311"}
Jul 13 19:18:03.266349 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:03.266252    2569 scope.go:117] "RemoveContainer" containerID="36377bd97a1ad6e2ad629d5a05774fba7b410fc2c1edbc2bb68e4be07f84997c"
Jul 13 19:18:03.293667 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:03.293622    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"]
Jul 13 19:18:03.301757 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:03.301728    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-kpd2m"]
Jul 13 19:18:03.351064 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:03.351025    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c" path="/var/lib/kubelet/pods/917138d3-6997-41e2-b523-80cceb9d1c5c/volumes"
Jul 13 19:18:26.552986 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:26.552953    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:18:39.643689 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:39.643638    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:18:42.536398 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.536366    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"]
Jul 13 19:18:42.536810 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.536723    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerName="revoke-keys"
Jul 13 19:18:42.536810 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.536733    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerName="revoke-keys"
Jul 13 19:18:42.536810 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.536744    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerName="revoke-keys"
Jul 13 19:18:42.536810 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.536750    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerName="revoke-keys"
Jul 13 19:18:42.536943 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.536818    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerName="revoke-keys"
Jul 13 19:18:42.536943 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.536829    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerName="revoke-keys"
Jul 13 19:18:42.539905 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.539885    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"
Jul 13 19:18:42.543213 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.543194    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:18:42.549130 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.549104    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"]
Jul 13 19:18:42.659147 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.659104    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d576591-5844-49d7-94a1-13f57c69d3f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-hjs7g\" (UID: \"5d576591-5844-49d7-94a1-13f57c69d3f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"
Jul 13 19:18:42.760498 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.760467    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d576591-5844-49d7-94a1-13f57c69d3f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-hjs7g\" (UID: \"5d576591-5844-49d7-94a1-13f57c69d3f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"
Jul 13 19:18:42.761074 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.761053    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d576591-5844-49d7-94a1-13f57c69d3f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-hjs7g\" (UID: \"5d576591-5844-49d7-94a1-13f57c69d3f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"
Jul 13 19:18:42.851323 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.851220    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"
Jul 13 19:18:42.995735 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:42.995707    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"]
Jul 13 19:18:42.998189 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:18:42.998163    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d576591_5844_49d7_94a1_13f57c69d3f9.slice/crio-b8a1d731b57ddca9af5d13e0c77f710aa721894aece01ba5874e6d76a183f197 WatchSource:0}: Error finding container b8a1d731b57ddca9af5d13e0c77f710aa721894aece01ba5874e6d76a183f197: Status 404 returned error can't find the container with id b8a1d731b57ddca9af5d13e0c77f710aa721894aece01ba5874e6d76a183f197
Jul 13 19:18:43.486162 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:43.486127    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g" event={"ID":"5d576591-5844-49d7-94a1-13f57c69d3f9","Type":"ContainerStarted","Data":"e68af4d72355985a89a8d2c424fc2742066f119701e815ea633ddcaa718f9bf8"}
Jul 13 19:18:43.486357 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:43.486191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g" event={"ID":"5d576591-5844-49d7-94a1-13f57c69d3f9","Type":"ContainerStarted","Data":"b8a1d731b57ddca9af5d13e0c77f710aa721894aece01ba5874e6d76a183f197"}
Jul 13 19:18:43.504596 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:43.504550    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g" podStartSLOduration=1.504534458 podStartE2EDuration="1.504534458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:18:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:18:43.502186379 +0000 UTC m=+2804.633474841" watchObservedRunningTime="2026-07-13 19:18:43.504534458 +0000 UTC m=+2804.635822917"
Jul 13 19:18:44.493275 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:44.493241    2569 generic.go:356] "Generic (PLEG): container finished" podID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerID="e68af4d72355985a89a8d2c424fc2742066f119701e815ea633ddcaa718f9bf8" exitCode=6
Jul 13 19:18:44.493680 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:44.493328    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g" event={"ID":"5d576591-5844-49d7-94a1-13f57c69d3f9","Type":"ContainerDied","Data":"e68af4d72355985a89a8d2c424fc2742066f119701e815ea633ddcaa718f9bf8"}
Jul 13 19:18:44.493680 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:44.493498    2569 scope.go:117] "RemoveContainer" containerID="e68af4d72355985a89a8d2c424fc2742066f119701e815ea633ddcaa718f9bf8"
Jul 13 19:18:45.501399 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:45.501361    2569 generic.go:356] "Generic (PLEG): container finished" podID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerID="e014edc3b2ac99b42ce88cbbf850d869b49b5e8fb890e5d986232b9c9829d6b8" exitCode=6
Jul 13 19:18:45.501968 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:45.501403    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g" event={"ID":"5d576591-5844-49d7-94a1-13f57c69d3f9","Type":"ContainerDied","Data":"e014edc3b2ac99b42ce88cbbf850d869b49b5e8fb890e5d986232b9c9829d6b8"}
Jul 13 19:18:45.501968 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:45.501465    2569 scope.go:117] "RemoveContainer" containerID="e68af4d72355985a89a8d2c424fc2742066f119701e815ea633ddcaa718f9bf8"
Jul 13 19:18:45.501968 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:45.501710    2569 scope.go:117] "RemoveContainer" containerID="e014edc3b2ac99b42ce88cbbf850d869b49b5e8fb890e5d986232b9c9829d6b8"
Jul 13 19:18:45.502148 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:18:45.501972    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-hjs7g_odh-ai-gateway-infra(5d576591-5844-49d7-94a1-13f57c69d3f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9"
Jul 13 19:18:46.508637 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:46.508607    2569 scope.go:117] "RemoveContainer" containerID="e014edc3b2ac99b42ce88cbbf850d869b49b5e8fb890e5d986232b9c9829d6b8"
Jul 13 19:18:46.509060 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:18:46.508798    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-hjs7g_odh-ai-gateway-infra(5d576591-5844-49d7-94a1-13f57c69d3f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9"
Jul 13 19:18:56.742555 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:56.742520    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:18:58.344486 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:58.344448    2569 scope.go:117] "RemoveContainer" containerID="e014edc3b2ac99b42ce88cbbf850d869b49b5e8fb890e5d986232b9c9829d6b8"
Jul 13 19:18:59.585054 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:59.585022    2569 generic.go:356] "Generic (PLEG): container finished" podID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerID="0ebdee9ba6362b8ac02f0988d75bd5e91e555b536f1ab53ae43a22371397ca34" exitCode=6
Jul 13 19:18:59.585518 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:59.585100    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g" event={"ID":"5d576591-5844-49d7-94a1-13f57c69d3f9","Type":"ContainerDied","Data":"0ebdee9ba6362b8ac02f0988d75bd5e91e555b536f1ab53ae43a22371397ca34"}
Jul 13 19:18:59.585518 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:59.585168    2569 scope.go:117] "RemoveContainer" containerID="e014edc3b2ac99b42ce88cbbf850d869b49b5e8fb890e5d986232b9c9829d6b8"
Jul 13 19:18:59.585518 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:18:59.585511    2569 scope.go:117] "RemoveContainer" containerID="0ebdee9ba6362b8ac02f0988d75bd5e91e555b536f1ab53ae43a22371397ca34"
Jul 13 19:18:59.585791 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:18:59.585771    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-hjs7g_odh-ai-gateway-infra(5d576591-5844-49d7-94a1-13f57c69d3f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9"
Jul 13 19:19:00.619884 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:00.619842    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"]
Jul 13 19:19:00.754479 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:00.754458    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"
Jul 13 19:19:00.927919 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:00.927823    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d576591-5844-49d7-94a1-13f57c69d3f9-maas-api-service-ca\") pod \"5d576591-5844-49d7-94a1-13f57c69d3f9\" (UID: \"5d576591-5844-49d7-94a1-13f57c69d3f9\") "
Jul 13 19:19:00.928230 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:00.928207    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d576591-5844-49d7-94a1-13f57c69d3f9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d576591-5844-49d7-94a1-13f57c69d3f9" (UID: "5d576591-5844-49d7-94a1-13f57c69d3f9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:19:01.029093 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:01.029059    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d576591-5844-49d7-94a1-13f57c69d3f9-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:01.599808 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:01.599782    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"
Jul 13 19:19:01.599981 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:01.599830    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g" event={"ID":"5d576591-5844-49d7-94a1-13f57c69d3f9","Type":"ContainerDied","Data":"b8a1d731b57ddca9af5d13e0c77f710aa721894aece01ba5874e6d76a183f197"}
Jul 13 19:19:01.599981 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:01.599887    2569 scope.go:117] "RemoveContainer" containerID="0ebdee9ba6362b8ac02f0988d75bd5e91e555b536f1ab53ae43a22371397ca34"
Jul 13 19:19:01.616815 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:01.616789    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"]
Jul 13 19:19:01.621291 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:01.621271    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-hjs7g"]
Jul 13 19:19:03.350636 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:03.350603    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9" path="/var/lib/kubelet/pods/5d576591-5844-49d7-94a1-13f57c69d3f9/volumes"
Jul 13 19:19:15.597777 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.597739    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"]
Jul 13 19:19:15.598380 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.597974    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl" podUID="fb2a1046-a2cf-44a2-bbdf-0453c91e58dc" containerName="maas-api" containerID="cri-o://67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4" gracePeriod=30
Jul 13 19:19:15.627930 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.627902    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x"]
Jul 13 19:19:15.633229 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.633205    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7b883e-29732835-chm4x"]
Jul 13 19:19:15.848477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.848418    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:19:15.958345 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.958311    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79hxv\" (UniqueName: \"kubernetes.io/projected/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-kube-api-access-79hxv\") pod \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") "
Jul 13 19:19:15.958537 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.958362    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-maas-api-tls\") pod \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") "
Jul 13 19:19:15.958537 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.958404    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-openshift-service-ca\") pod \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\" (UID: \"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc\") "
Jul 13 19:19:15.958828 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.958798    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb2a1046-a2cf-44a2-bbdf-0453c91e58dc" (UID: "fb2a1046-a2cf-44a2-bbdf-0453c91e58dc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:19:15.960603 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.960578    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb2a1046-a2cf-44a2-bbdf-0453c91e58dc" (UID: "fb2a1046-a2cf-44a2-bbdf-0453c91e58dc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:19:15.960731 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:15.960677    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-kube-api-access-79hxv" (OuterVolumeSpecName: "kube-api-access-79hxv") pod "fb2a1046-a2cf-44a2-bbdf-0453c91e58dc" (UID: "fb2a1046-a2cf-44a2-bbdf-0453c91e58dc"). InnerVolumeSpecName "kube-api-access-79hxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:19:16.059063 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.059034    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:16.059063 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.059060    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:16.059063 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.059070    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79hxv\" (UniqueName: \"kubernetes.io/projected/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc-kube-api-access-79hxv\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:16.687063 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.687030    2569 generic.go:356] "Generic (PLEG): container finished" podID="fb2a1046-a2cf-44a2-bbdf-0453c91e58dc" containerID="67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4" exitCode=0
Jul 13 19:19:16.687446 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.687088    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"
Jul 13 19:19:16.687446 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.687119    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl" event={"ID":"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc","Type":"ContainerDied","Data":"67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4"}
Jul 13 19:19:16.687446 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.687176    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl" event={"ID":"fb2a1046-a2cf-44a2-bbdf-0453c91e58dc","Type":"ContainerDied","Data":"cc9747c839a1356352a50c5ad699ed0a62da5d786907822b411f8f8c79ed0865"}
Jul 13 19:19:16.687446 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.687205    2569 scope.go:117] "RemoveContainer" containerID="67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4"
Jul 13 19:19:16.696822 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.696805    2569 scope.go:117] "RemoveContainer" containerID="67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4"
Jul 13 19:19:16.697066 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:19:16.697049    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4\": container with ID starting with 67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4 not found: ID does not exist" containerID="67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4"
Jul 13 19:19:16.697113 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.697074    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4"} err="failed to get container status \"67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4\": rpc error: code = NotFound desc = could not find container \"67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4\": container with ID starting with 67b019b0a28d7289de0204c5469b66d99f6aa05738fd7ec60dbd994db08824b4 not found: ID does not exist"
Jul 13 19:19:16.709222 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.709198    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"]
Jul 13 19:19:16.713288 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:16.713267    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b883e-7b4c945797-s9hhl"]
Jul 13 19:19:17.351287 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:17.351253    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="652d647f-b3d6-455e-b506-ac85dd297026" path="/var/lib/kubelet/pods/652d647f-b3d6-455e-b506-ac85dd297026/volumes"
Jul 13 19:19:17.351617 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:17.351604    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb2a1046-a2cf-44a2-bbdf-0453c91e58dc" path="/var/lib/kubelet/pods/fb2a1046-a2cf-44a2-bbdf-0453c91e58dc/volumes"
Jul 13 19:19:19.036952 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:19.036921    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:19:30.911384 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:30.911354    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"]
Jul 13 19:19:30.911791 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:30.911567    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy" containerID="cri-o://db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc" gracePeriod=30
Jul 13 19:19:31.292296 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292259    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m"]
Jul 13 19:19:31.292629 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292615    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb2a1046-a2cf-44a2-bbdf-0453c91e58dc" containerName="maas-api"
Jul 13 19:19:31.292718 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292631    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb2a1046-a2cf-44a2-bbdf-0453c91e58dc" containerName="maas-api"
Jul 13 19:19:31.292718 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292641    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerName="revoke-keys"
Jul 13 19:19:31.292718 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292661    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerName="revoke-keys"
Jul 13 19:19:31.292718 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292679    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerName="revoke-keys"
Jul 13 19:19:31.292718 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292685    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerName="revoke-keys"
Jul 13 19:19:31.292718 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292695    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerName="revoke-keys"
Jul 13 19:19:31.292718 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292700    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerName="revoke-keys"
Jul 13 19:19:31.292938 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292749    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerName="revoke-keys"
Jul 13 19:19:31.292938 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292756    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb2a1046-a2cf-44a2-bbdf-0453c91e58dc" containerName="maas-api"
Jul 13 19:19:31.292938 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292764    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerName="revoke-keys"
Jul 13 19:19:31.292938 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.292771    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="917138d3-6997-41e2-b523-80cceb9d1c5c" containerName="revoke-keys"
Jul 13 19:19:31.297225 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.297207    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m"
Jul 13 19:19:31.300911 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.300888    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:19:31.311039 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.311016    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m"]
Jul 13 19:19:31.395844 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.395813    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e91902-5499m\" (UID: \"f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m"
Jul 13 19:19:31.497330 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.497292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e91902-5499m\" (UID: \"f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m"
Jul 13 19:19:31.498074 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.498048    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e91902-5499m\" (UID: \"f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m"
Jul 13 19:19:31.607309 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.607225    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m"
Jul 13 19:19:31.748189 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.748159    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m"]
Jul 13 19:19:31.749969 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:19:31.749944    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7351d9d_cfa9_4954_aa2a_4b0ac6b8961a.slice/crio-3a0414423a060f16cecea06fbd0a284fd0e28f4a566e0bbd6116bbbcbae263e2 WatchSource:0}: Error finding container 3a0414423a060f16cecea06fbd0a284fd0e28f4a566e0bbd6116bbbcbae263e2: Status 404 returned error can't find the container with id 3a0414423a060f16cecea06fbd0a284fd0e28f4a566e0bbd6116bbbcbae263e2
Jul 13 19:19:31.771247 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:31.771218    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m" event={"ID":"f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a","Type":"ContainerStarted","Data":"3a0414423a060f16cecea06fbd0a284fd0e28f4a566e0bbd6116bbbcbae263e2"}
Jul 13 19:19:32.778395 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:32.778363    2569 generic.go:356] "Generic (PLEG): container finished" podID="f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a" containerID="8c4f57e0d29332947b834bcefc9c8c751667a6b132dba52befbae03e5cc4aad5" exitCode=0
Jul 13 19:19:32.778811 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:32.778449    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m" event={"ID":"f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a","Type":"ContainerDied","Data":"8c4f57e0d29332947b834bcefc9c8c751667a6b132dba52befbae03e5cc4aad5"}
Jul 13 19:19:33.159975 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:33.159889    2569 patch_prober.go:28] interesting pod/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 13 19:19:33.159975 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:33.159952    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 19:19:33.926956 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:33.926931    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m"
Jul 13 19:19:34.021070 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:34.021037    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a-maas-api-service-ca\") pod \"f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a\" (UID: \"f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a\") "
Jul 13 19:19:34.021411 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:34.021388    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a" (UID: "f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:19:34.122705 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:34.122609    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a-maas-api-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:34.797493 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:34.797458    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m" event={"ID":"f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a","Type":"ContainerDied","Data":"3a0414423a060f16cecea06fbd0a284fd0e28f4a566e0bbd6116bbbcbae263e2"}
Jul 13 19:19:34.797493 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:34.797486    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e91902-5499m"
Jul 13 19:19:34.797758 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:34.797511    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a0414423a060f16cecea06fbd0a284fd0e28f4a566e0bbd6116bbbcbae263e2"
Jul 13 19:19:36.162805 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.162772    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:19:36.242774 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.242729    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-token\") pod \"ee25864f-6bd9-4d16-8310-7c5f794df9da\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") "
Jul 13 19:19:36.242964 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.242806    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-data\") pod \"ee25864f-6bd9-4d16-8310-7c5f794df9da\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") "
Jul 13 19:19:36.242964 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.242912    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-envoy\") pod \"ee25864f-6bd9-4d16-8310-7c5f794df9da\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") "
Jul 13 19:19:36.242964 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.242950    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blvb5\" (UniqueName: \"kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-kube-api-access-blvb5\") pod \"ee25864f-6bd9-4d16-8310-7c5f794df9da\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") "
Jul 13 19:19:36.243134 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.242986    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-certs\") pod \"ee25864f-6bd9-4d16-8310-7c5f794df9da\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") "
Jul 13 19:19:36.243134 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.243099    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-credential-socket\") pod \"ee25864f-6bd9-4d16-8310-7c5f794df9da\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") "
Jul 13 19:19:36.243237 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.243166    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-socket\") pod \"ee25864f-6bd9-4d16-8310-7c5f794df9da\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") "
Jul 13 19:19:36.243237 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.243197    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-podinfo\") pod \"ee25864f-6bd9-4d16-8310-7c5f794df9da\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") "
Jul 13 19:19:36.243237 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.243223    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee25864f-6bd9-4d16-8310-7c5f794df9da" (UID: "ee25864f-6bd9-4d16-8310-7c5f794df9da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:19:36.243237 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.243231    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee25864f-6bd9-4d16-8310-7c5f794df9da-istiod-ca-cert\") pod \"ee25864f-6bd9-4d16-8310-7c5f794df9da\" (UID: \"ee25864f-6bd9-4d16-8310-7c5f794df9da\") "
Jul 13 19:19:36.243426 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.243386    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee25864f-6bd9-4d16-8310-7c5f794df9da" (UID: "ee25864f-6bd9-4d16-8310-7c5f794df9da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:19:36.243761 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.243557    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:36.243761 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.243579    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-workload-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:36.243967 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.243912    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee25864f-6bd9-4d16-8310-7c5f794df9da" (UID: "ee25864f-6bd9-4d16-8310-7c5f794df9da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:19:36.244033 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.244013    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee25864f-6bd9-4d16-8310-7c5f794df9da-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee25864f-6bd9-4d16-8310-7c5f794df9da" (UID: "ee25864f-6bd9-4d16-8310-7c5f794df9da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:19:36.244129 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.244103    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee25864f-6bd9-4d16-8310-7c5f794df9da" (UID: "ee25864f-6bd9-4d16-8310-7c5f794df9da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:19:36.245200 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.245169    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee25864f-6bd9-4d16-8310-7c5f794df9da" (UID: "ee25864f-6bd9-4d16-8310-7c5f794df9da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:19:36.245295 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.245255    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee25864f-6bd9-4d16-8310-7c5f794df9da" (UID: "ee25864f-6bd9-4d16-8310-7c5f794df9da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 19:19:36.245806 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.245780    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-kube-api-access-blvb5" (OuterVolumeSpecName: "kube-api-access-blvb5") pod "ee25864f-6bd9-4d16-8310-7c5f794df9da" (UID: "ee25864f-6bd9-4d16-8310-7c5f794df9da"). InnerVolumeSpecName "kube-api-access-blvb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:19:36.245869 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.245848    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee25864f-6bd9-4d16-8310-7c5f794df9da" (UID: "ee25864f-6bd9-4d16-8310-7c5f794df9da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 19:19:36.344309 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.344273    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-podinfo\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:36.344309 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.344302    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee25864f-6bd9-4d16-8310-7c5f794df9da-istiod-ca-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:36.344309 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.344311    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-token\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:36.344309 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.344319    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-data\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:36.344580 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.344326    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-istio-envoy\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:36.344580 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.344334    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blvb5\" (UniqueName: \"kubernetes.io/projected/ee25864f-6bd9-4d16-8310-7c5f794df9da-kube-api-access-blvb5\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:36.344580 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.344343    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee25864f-6bd9-4d16-8310-7c5f794df9da-credential-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:36.810064 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.810031    2569 generic.go:356] "Generic (PLEG): container finished" podID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerID="db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc" exitCode=0
Jul 13 19:19:36.810267 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.810074    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"
Jul 13 19:19:36.810267 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.810119    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" event={"ID":"ee25864f-6bd9-4d16-8310-7c5f794df9da","Type":"ContainerDied","Data":"db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc"}
Jul 13 19:19:36.810267 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.810178    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv" event={"ID":"ee25864f-6bd9-4d16-8310-7c5f794df9da","Type":"ContainerDied","Data":"54997a26826a4e58a2c58e252ebcc34f4d28c08c1f52e3821c9df2972a8d8cde"}
Jul 13 19:19:36.810267 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.810209    2569 scope.go:117] "RemoveContainer" containerID="db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc"
Jul 13 19:19:36.820319 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.820301    2569 scope.go:117] "RemoveContainer" containerID="efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c"
Jul 13 19:19:36.828496 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.828480    2569 scope.go:117] "RemoveContainer" containerID="db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc"
Jul 13 19:19:36.828775 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:19:36.828758    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc\": container with ID starting with db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc not found: ID does not exist" containerID="db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc"
Jul 13 19:19:36.828818 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.828783    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc"} err="failed to get container status \"db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc\": rpc error: code = NotFound desc = could not find container \"db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc\": container with ID starting with db275a154c44a84378203bf596d2ca1c07117218945dce9617da2494608ed6bc not found: ID does not exist"
Jul 13 19:19:36.828818 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.828800    2569 scope.go:117] "RemoveContainer" containerID="efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c"
Jul 13 19:19:36.829035 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:19:36.829019    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c\": container with ID starting with efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c not found: ID does not exist" containerID="efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c"
Jul 13 19:19:36.829076 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.829042    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c"} err="failed to get container status \"efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c\": rpc error: code = NotFound desc = could not find container \"efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c\": container with ID starting with efaa75648e0a07596b2c8b9d73202bb94d022f275af94cced3d5750aa294777c not found: ID does not exist"
Jul 13 19:19:36.834032 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.834011    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"]
Jul 13 19:19:36.840402 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:36.840379    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-7b883e-openshift-default-5fddc79979-vh5dv"]
Jul 13 19:19:37.350861 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:37.350828    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" path="/var/lib/kubelet/pods/ee25864f-6bd9-4d16-8310-7c5f794df9da/volumes"
Jul 13 19:19:41.764046 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:41.764009    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"]
Jul 13 19:19:41.764487 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:41.764269    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6" podUID="5bcf65a7-f5e6-4b33-8025-a1906106687d" containerName="maas-api" containerID="cri-o://2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01" gracePeriod=30
Jul 13 19:19:41.797382 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:41.797352    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc"]
Jul 13 19:19:41.801917 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:41.801891    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e91902-29732835-t8fjc"]
Jul 13 19:19:42.018536 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.018474    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:19:42.089748 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.089713    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srrm6\" (UniqueName: \"kubernetes.io/projected/5bcf65a7-f5e6-4b33-8025-a1906106687d-kube-api-access-srrm6\") pod \"5bcf65a7-f5e6-4b33-8025-a1906106687d\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") "
Jul 13 19:19:42.089917 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.089804    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bcf65a7-f5e6-4b33-8025-a1906106687d-openshift-service-ca\") pod \"5bcf65a7-f5e6-4b33-8025-a1906106687d\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") "
Jul 13 19:19:42.089917 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.089850    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcf65a7-f5e6-4b33-8025-a1906106687d-maas-api-tls\") pod \"5bcf65a7-f5e6-4b33-8025-a1906106687d\" (UID: \"5bcf65a7-f5e6-4b33-8025-a1906106687d\") "
Jul 13 19:19:42.090193 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.090171    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bcf65a7-f5e6-4b33-8025-a1906106687d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5bcf65a7-f5e6-4b33-8025-a1906106687d" (UID: "5bcf65a7-f5e6-4b33-8025-a1906106687d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 19:19:42.092059 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.092032    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bcf65a7-f5e6-4b33-8025-a1906106687d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5bcf65a7-f5e6-4b33-8025-a1906106687d" (UID: "5bcf65a7-f5e6-4b33-8025-a1906106687d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 19:19:42.092171 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.092060    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bcf65a7-f5e6-4b33-8025-a1906106687d-kube-api-access-srrm6" (OuterVolumeSpecName: "kube-api-access-srrm6") pod "5bcf65a7-f5e6-4b33-8025-a1906106687d" (UID: "5bcf65a7-f5e6-4b33-8025-a1906106687d"). InnerVolumeSpecName "kube-api-access-srrm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 19:19:42.191343 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.191308    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcf65a7-f5e6-4b33-8025-a1906106687d-maas-api-tls\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:42.191343 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.191337    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srrm6\" (UniqueName: \"kubernetes.io/projected/5bcf65a7-f5e6-4b33-8025-a1906106687d-kube-api-access-srrm6\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:42.191343 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.191347    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bcf65a7-f5e6-4b33-8025-a1906106687d-openshift-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\""
Jul 13 19:19:42.843396 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.843359    2569 generic.go:356] "Generic (PLEG): container finished" podID="5bcf65a7-f5e6-4b33-8025-a1906106687d" containerID="2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01" exitCode=0
Jul 13 19:19:42.843890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.843408    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"
Jul 13 19:19:42.843890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.843409    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6" event={"ID":"5bcf65a7-f5e6-4b33-8025-a1906106687d","Type":"ContainerDied","Data":"2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01"}
Jul 13 19:19:42.843890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.843542    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6" event={"ID":"5bcf65a7-f5e6-4b33-8025-a1906106687d","Type":"ContainerDied","Data":"47b5de9508cd959e9be3e1fda2e69219588e4fd5820161120d60ea99b8793dd2"}
Jul 13 19:19:42.843890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.843574    2569 scope.go:117] "RemoveContainer" containerID="2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01"
Jul 13 19:19:42.853121 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.853105    2569 scope.go:117] "RemoveContainer" containerID="2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01"
Jul 13 19:19:42.853387 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:19:42.853355    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01\": container with ID starting with 2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01 not found: ID does not exist" containerID="2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01"
Jul 13 19:19:42.853431 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.853393    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01"} err="failed to get container status \"2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01\": rpc error: code = NotFound desc = could not find container \"2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01\": container with ID starting with 2e4792c6fa4d1a235c98e9f5d42ad2c182584c36199da8e01c606fd59cfaba01 not found: ID does not exist"
Jul 13 19:19:42.865156 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.865132    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"]
Jul 13 19:19:42.870483 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:42.870463    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e91902-7856d97f9c-xtjz6"]
Jul 13 19:19:43.350522 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:43.350487    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bcf65a7-f5e6-4b33-8025-a1906106687d" path="/var/lib/kubelet/pods/5bcf65a7-f5e6-4b33-8025-a1906106687d/volumes"
Jul 13 19:19:43.350825 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:43.350811    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bfd0ca9-48b1-438b-8e35-ece3355a4a45" path="/var/lib/kubelet/pods/9bfd0ca9-48b1-438b-8e35-ece3355a4a45/volumes"
Jul 13 19:19:45.039019 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:19:45.038982    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ghhd9"]
Jul 13 19:20:02.893879 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:02.893853    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4wqcl_3693db18-1d4a-4c46-8cb9-75ba5c992a83/manager/0.log"
Jul 13 19:20:03.011706 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:03.011668    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zls45_5ce2c5df-34db-4e6f-861e-a080808a2530/manager/0.log"
Jul 13 19:20:03.279061 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:03.279033    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rcfd2_bb9b384f-4d45-4297-8e1c-94d06747ed00/server/0.log"
Jul 13 19:20:03.394028 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:03.394004    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hfcwh_80269e47-5ba4-4b0a-b925-06cbf69049bd/manager/0.log"
Jul 13 19:20:03.643031 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:03.642954    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4789cfbf-t92zq_76011d1d-de12-4769-b91f-e5600b77090a/manager/0.log"
Jul 13 19:20:03.652794 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:03.652772    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4789cfbf-t92zq_76011d1d-de12-4769-b91f-e5600b77090a/copy-manifests/0.log"
Jul 13 19:20:04.497379 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:04.497355    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732835-nl7hh_5de82b66-f389-4dd4-911d-3d725b9849f4/cleanup/0.log"
Jul 13 19:20:04.846202 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:04.846119    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-e196f1-ghsgd_5585bf3c-8ba4-4244-93fb-790cfc3019d7/revoke-keys/0.log"
Jul 13 19:20:04.954364 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:04.954338    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-5ffdec-plh2q_427d5b7f-58d8-4ef4-9ea2-0cef94b5cd29/revoke-keys/0.log"
Jul 13 19:20:05.177477 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:05.177393    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-5d44da-qsfg7_0a1aa635-6483-4970-bfd9-bb8eda99ecbe/revoke-keys/0.log"
Jul 13 19:20:05.290926 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:05.290902    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-0a46d9-r5t2c_02d3e594-bc5a-4f4e-b1d1-25d0d045bf3a/revoke-keys/0.log"
Jul 13 19:20:05.403964 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:05.403941    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-7577a6d1-wpcgq_f35d0c37-6215-4bfb-9344-fdca984d2e55/revoke-keys/0.log"
Jul 13 19:20:05.518836 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:05.518812    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-991a1835-fltkj_a18adf4b-9dd1-4df9-840c-46e1fb812492/revoke-keys/0.log"
Jul 13 19:20:05.759764 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:05.759734    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-e91902-5499m_f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a/revoke-keys/0.log"
Jul 13 19:20:06.634154 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:06.634130    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp_25a50f5b-09ad-48b7-9df1-08657541b55e/util/0.log"
Jul 13 19:20:06.640912 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:06.640886    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp_25a50f5b-09ad-48b7-9df1-08657541b55e/pull/0.log"
Jul 13 19:20:06.646857 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:06.646837    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp_25a50f5b-09ad-48b7-9df1-08657541b55e/extract/0.log"
Jul 13 19:20:06.768236 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:06.768210    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x_83168f79-3b2b-402e-8f90-6c712e45c39d/extract/0.log"
Jul 13 19:20:06.785687 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:06.785637    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x_83168f79-3b2b-402e-8f90-6c712e45c39d/util/0.log"
Jul 13 19:20:06.794254 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:06.794238    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x_83168f79-3b2b-402e-8f90-6c712e45c39d/pull/0.log"
Jul 13 19:20:06.907132 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:06.907062    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z_bfef29e3-76c2-4926-9db0-764c65cc63cd/util/0.log"
Jul 13 19:20:06.917045 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:06.917019    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z_bfef29e3-76c2-4926-9db0-764c65cc63cd/pull/0.log"
Jul 13 19:20:06.926224 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:06.926201    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z_bfef29e3-76c2-4926-9db0-764c65cc63cd/extract/0.log"
Jul 13 19:20:07.033560 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:07.033531    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r_7da49cce-9637-428c-a819-f878fb853e3e/util/0.log"
Jul 13 19:20:07.043582 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:07.043558    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r_7da49cce-9637-428c-a819-f878fb853e3e/pull/0.log"
Jul 13 19:20:07.051624 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:07.051595    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r_7da49cce-9637-428c-a819-f878fb853e3e/extract/0.log"
Jul 13 19:20:07.288083 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:07.288053    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jwdcw_56c20e66-c182-47d1-9615-f15ca870e130/manager/0.log"
Jul 13 19:20:07.902953 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:07.902928    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ghhd9_98919ddd-2764-452c-958c-72fda9522a34/limitador/0.log"
Jul 13 19:20:08.501457 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:08.501425    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6jt48_ba045cfc-ce28-452c-9bfc-f9d30a84d1fb/discovery/0.log"
Jul 13 19:20:08.739840 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:08.739809    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f8768797-p9r57_f0ab603f-cc81-4023-b68f-75c3bbdcaed7/kube-auth-proxy/0.log"
Jul 13 19:20:09.227005 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:09.226975    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-597d7847fb-55tbz_f5c0e7af-b62f-4dfc-810f-39223c968ce9/router/0.log"
Jul 13 19:20:10.058393 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:10.058363    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-m9n8b_43a08093-5986-4d29-adc7-ae2be5201c98/main/0.log"
Jul 13 19:20:17.500992 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:17.500961    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ffsc6_8fe11421-9613-4ca6-8c39-c44b3682f360/global-pull-secret-syncer/0.log"
Jul 13 19:20:17.636633 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:17.636603    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9fx5s_edb49c7e-ea32-4018-9f70-6d25d8b680c4/konnectivity-agent/0.log"
Jul 13 19:20:17.750084 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:17.750048    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-206.ec2.internal_48897f40f8f0e850d106dd2f49700b6a/haproxy/0.log"
Jul 13 19:20:21.088573 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.088536    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp_25a50f5b-09ad-48b7-9df1-08657541b55e/extract/0.log"
Jul 13 19:20:21.113335 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.113312    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp_25a50f5b-09ad-48b7-9df1-08657541b55e/util/0.log"
Jul 13 19:20:21.142021 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.142000    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jljfp_25a50f5b-09ad-48b7-9df1-08657541b55e/pull/0.log"
Jul 13 19:20:21.186259 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.186237    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x_83168f79-3b2b-402e-8f90-6c712e45c39d/extract/0.log"
Jul 13 19:20:21.227923 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.227899    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x_83168f79-3b2b-402e-8f90-6c712e45c39d/util/0.log"
Jul 13 19:20:21.252324 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.252303    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wng4x_83168f79-3b2b-402e-8f90-6c712e45c39d/pull/0.log"
Jul 13 19:20:21.282490 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.282466    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z_bfef29e3-76c2-4926-9db0-764c65cc63cd/extract/0.log"
Jul 13 19:20:21.334046 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.334023    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z_bfef29e3-76c2-4926-9db0-764c65cc63cd/util/0.log"
Jul 13 19:20:21.359471 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.359399    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnn9z_bfef29e3-76c2-4926-9db0-764c65cc63cd/pull/0.log"
Jul 13 19:20:21.387666 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.387637    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r_7da49cce-9637-428c-a819-f878fb853e3e/extract/0.log"
Jul 13 19:20:21.410289 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.410264    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r_7da49cce-9637-428c-a819-f878fb853e3e/util/0.log"
Jul 13 19:20:21.449374 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.449349    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dkw7r_7da49cce-9637-428c-a819-f878fb853e3e/pull/0.log"
Jul 13 19:20:21.810439 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:21.810412    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jwdcw_56c20e66-c182-47d1-9615-f15ca870e130/manager/0.log"
Jul 13 19:20:22.097798 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:22.097711    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ghhd9_98919ddd-2764-452c-958c-72fda9522a34/limitador/0.log"
Jul 13 19:20:24.077547 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:24.077523    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-hrjpr_0b35e240-a9b7-4d64-be6b-bb32082a2073/cluster-monitoring-operator/0.log"
Jul 13 19:20:24.259826 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:24.259793    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6vctn_feb9153f-0b3c-4777-9781-9132d572cf39/node-exporter/0.log"
Jul 13 19:20:24.286879 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:24.286852    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6vctn_feb9153f-0b3c-4777-9781-9132d572cf39/kube-rbac-proxy/0.log"
Jul 13 19:20:24.311027 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:24.311005    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6vctn_feb9153f-0b3c-4777-9781-9132d572cf39/init-textfile/0.log"
Jul 13 19:20:27.819710 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.819670    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"]
Jul 13 19:20:27.820065 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820028    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bcf65a7-f5e6-4b33-8025-a1906106687d" containerName="maas-api"
Jul 13 19:20:27.820065 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820041    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bcf65a7-f5e6-4b33-8025-a1906106687d" containerName="maas-api"
Jul 13 19:20:27.820065 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820052    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy"
Jul 13 19:20:27.820065 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820057    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy"
Jul 13 19:20:27.820065 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820065    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy"
Jul 13 19:20:27.820239 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820071    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy"
Jul 13 19:20:27.820239 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820082    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a" containerName="revoke-keys"
Jul 13 19:20:27.820239 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820099    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a" containerName="revoke-keys"
Jul 13 19:20:27.820239 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820114    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerName="revoke-keys"
Jul 13 19:20:27.820239 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820119    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerName="revoke-keys"
Jul 13 19:20:27.820239 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820173    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy"
Jul 13 19:20:27.820239 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820181    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee25864f-6bd9-4d16-8310-7c5f794df9da" containerName="istio-proxy"
Jul 13 19:20:27.820239 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820188    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d576591-5844-49d7-94a1-13f57c69d3f9" containerName="revoke-keys"
Jul 13 19:20:27.820239 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820193    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7351d9d-cfa9-4954-aa2a-4b0ac6b8961a" containerName="revoke-keys"
Jul 13 19:20:27.820239 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.820198    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bcf65a7-f5e6-4b33-8025-a1906106687d" containerName="maas-api"
Jul 13 19:20:27.823365 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.823349    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.827282 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.827257    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-podres\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.827448 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.827292    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-sys\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.827448 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.827344    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-proc\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.827448 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.827378    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2zwf\" (UniqueName: \"kubernetes.io/projected/dbe57ef5-1096-48f0-98d8-20cda1da3250-kube-api-access-j2zwf\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.827448 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.827412    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-lib-modules\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.828388 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.828358    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-msvxh\"/\"kube-root-ca.crt\""
Jul 13 19:20:27.828952 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.828931    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-msvxh\"/\"default-dockercfg-tkvs5\""
Jul 13 19:20:27.829382 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.829362    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-msvxh\"/\"openshift-service-ca.crt\""
Jul 13 19:20:27.831930 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.831908    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"]
Jul 13 19:20:27.928062 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.928020    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-podres\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.928257 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.928077    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-sys\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.928257 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.928110    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-proc\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.928257 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.928133    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2zwf\" (UniqueName: \"kubernetes.io/projected/dbe57ef5-1096-48f0-98d8-20cda1da3250-kube-api-access-j2zwf\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.928257 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.928150    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-lib-modules\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.928257 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.928192    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-podres\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.928257 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.928208    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-proc\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.928257 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.928208    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-sys\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.928591 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.928290    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbe57ef5-1096-48f0-98d8-20cda1da3250-lib-modules\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.938832 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.938812    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2zwf\" (UniqueName: \"kubernetes.io/projected/dbe57ef5-1096-48f0-98d8-20cda1da3250-kube-api-access-j2zwf\") pod \"perf-node-gather-daemonset-cngvz\" (UID: \"dbe57ef5-1096-48f0-98d8-20cda1da3250\") " pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:27.975939 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:27.975912    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-86dfbf7594-92rxg_5a8681f7-ef88-44c5-9b99-bdaad3bf79b0/console/0.log"
Jul 13 19:20:28.133923 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:28.133836    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:28.265406 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:28.265381    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"]
Jul 13 19:20:28.266995 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:20:28.266961    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddbe57ef5_1096_48f0_98d8_20cda1da3250.slice/crio-c8c4fa854d663d7882b6784776a717272f9ce597ccf0124b3342cd4136ce3659 WatchSource:0}: Error finding container c8c4fa854d663d7882b6784776a717272f9ce597ccf0124b3342cd4136ce3659: Status 404 returned error can't find the container with id c8c4fa854d663d7882b6784776a717272f9ce597ccf0124b3342cd4136ce3659
Jul 13 19:20:29.097172 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:29.097138    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz" event={"ID":"dbe57ef5-1096-48f0-98d8-20cda1da3250","Type":"ContainerStarted","Data":"53fbcce2ccfbfd5d24d075a76a86cefabed295afe6bacccc61a731f7e9f4f13e"}
Jul 13 19:20:29.097627 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:29.097195    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz" event={"ID":"dbe57ef5-1096-48f0-98d8-20cda1da3250","Type":"ContainerStarted","Data":"c8c4fa854d663d7882b6784776a717272f9ce597ccf0124b3342cd4136ce3659"}
Jul 13 19:20:29.097627 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:29.097216    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:29.118582 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:29.118531    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz" podStartSLOduration=2.118516865 podStartE2EDuration="2.118516865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:20:29.11782423 +0000 UTC m=+2910.249112691" watchObservedRunningTime="2026-07-13 19:20:29.118516865 +0000 UTC m=+2910.249805324"
Jul 13 19:20:29.812057 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:29.812024    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s8dct_60f2abdb-0f87-4d7b-abdd-b9dfef4eb951/dns/0.log"
Jul 13 19:20:29.846646 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:29.846621    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s8dct_60f2abdb-0f87-4d7b-abdd-b9dfef4eb951/kube-rbac-proxy/0.log"
Jul 13 19:20:30.058597 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:30.058567    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-88fdd_91a5e102-1948-414f-bfc4-a468dfe9f3ba/dns-node-resolver/0.log"
Jul 13 19:20:30.713877 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:30.713841    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-bd58b54dc-59p46_d47f9940-79d2-41a7-bc33-bbb53cc5b89b/registry/0.log"
Jul 13 19:20:30.844495 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:30.844457    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qs9wn_d153a23e-6499-4764-ad85-666ef8b2dc72/node-ca/0.log"
Jul 13 19:20:32.025197 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.025167    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6jt48_ba045cfc-ce28-452c-9bfc-f9d30a84d1fb/discovery/0.log"
Jul 13 19:20:32.081570 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.081546    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f8768797-p9r57_f0ab603f-cc81-4023-b68f-75c3bbdcaed7/kube-auth-proxy/0.log"
Jul 13 19:20:32.347040 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.346954    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-597d7847fb-55tbz_f5c0e7af-b62f-4dfc-810f-39223c968ce9/router/0.log"
Jul 13 19:20:32.647816 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.647738    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx"]
Jul 13 19:20:32.650869 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.650849    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx"
Jul 13 19:20:32.653567 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.653547    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 19:20:32.653675 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.653606    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lkrf8\""
Jul 13 19:20:32.659920 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.659896    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx"]
Jul 13 19:20:32.661609 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.661583    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-k5hmx\" (UID: \"b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx"
Jul 13 19:20:32.762918 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.762879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-k5hmx\" (UID: \"b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx"
Jul 13 19:20:32.763475 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.763456    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78327326-k5hmx\" (UID: \"b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx"
Jul 13 19:20:32.962556 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:32.962471    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx"
Jul 13 19:20:33.084196 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:33.084171    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nlnzn_d1729c34-3c2c-4fa7-a94e-f7628fad363e/serve-healthcheck-canary/0.log"
Jul 13 19:20:33.101140 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:33.101101    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx"]
Jul 13 19:20:33.104924 ip-10-0-139-206 kubenswrapper[2569]: W0713 19:20:33.104893    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4d3e704_19f3_45b4_b3f8_cd14a23bfdf2.slice/crio-ed1c72a9d04334141bbc9242e6177f24af35254421f6caf335d737b99803f1f3 WatchSource:0}: Error finding container ed1c72a9d04334141bbc9242e6177f24af35254421f6caf335d737b99803f1f3: Status 404 returned error can't find the container with id ed1c72a9d04334141bbc9242e6177f24af35254421f6caf335d737b99803f1f3
Jul 13 19:20:33.122666 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:33.122614    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx" event={"ID":"b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2","Type":"ContainerStarted","Data":"ed1c72a9d04334141bbc9242e6177f24af35254421f6caf335d737b99803f1f3"}
Jul 13 19:20:33.727337 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:33.727319    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kcsc8_cf38ff56-4be5-4117-814f-a64b1b5d44ea/kube-rbac-proxy/0.log"
Jul 13 19:20:33.752934 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:33.752913    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kcsc8_cf38ff56-4be5-4117-814f-a64b1b5d44ea/exporter/0.log"
Jul 13 19:20:33.777128 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:33.777106    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kcsc8_cf38ff56-4be5-4117-814f-a64b1b5d44ea/extractor/0.log"
Jul 13 19:20:34.129256 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:34.129224    2569 generic.go:356] "Generic (PLEG): container finished" podID="b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2" containerID="e9e267a65dcd6ddbccfe09b89102cbffa4aecc586b083ed97848b4f53698efb5" exitCode=6
Jul 13 19:20:34.129687 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:34.129313    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx" event={"ID":"b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2","Type":"ContainerDied","Data":"e9e267a65dcd6ddbccfe09b89102cbffa4aecc586b083ed97848b4f53698efb5"}
Jul 13 19:20:34.129687 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:34.129493    2569 scope.go:117] "RemoveContainer" containerID="e9e267a65dcd6ddbccfe09b89102cbffa4aecc586b083ed97848b4f53698efb5"
Jul 13 19:20:35.113334 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:35.113308    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-msvxh/perf-node-gather-daemonset-cngvz"
Jul 13 19:20:35.137829 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:35.137796    2569 generic.go:356] "Generic (PLEG): container finished" podID="b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2" containerID="99e9d8d08c42303c5f33ab937e81edfd5e0b1b7336669fdad30600cf03cd2a4d" exitCode=6
Jul 13 19:20:35.138196 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:35.137877    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx" event={"ID":"b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2","Type":"ContainerDied","Data":"99e9d8d08c42303c5f33ab937e81edfd5e0b1b7336669fdad30600cf03cd2a4d"}
Jul 13 19:20:35.138196 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:35.137939    2569 scope.go:117] "RemoveContainer" containerID="e9e267a65dcd6ddbccfe09b89102cbffa4aecc586b083ed97848b4f53698efb5"
Jul 13 19:20:35.138625 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:35.138208    2569 scope.go:117] "RemoveContainer" containerID="99e9d8d08c42303c5f33ab937e81edfd5e0b1b7336669fdad30600cf03cd2a4d"
Jul 13 19:20:35.138625 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:20:35.138391    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-k5hmx_odh-ai-gateway-infra(b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx" podUID="b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2"
Jul 13 19:20:36.144556 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:36.144530    2569 scope.go:117] "RemoveContainer" containerID="99e9d8d08c42303c5f33ab937e81edfd5e0b1b7336669fdad30600cf03cd2a4d"
Jul 13 19:20:36.144965 ip-10-0-139-206 kubenswrapper[2569]: E0713 19:20:36.144748    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-78327326-k5hmx_odh-ai-gateway-infra(b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78327326-k5hmx" podUID="b4d3e704-19f3-45b4-b3f8-cd14a23bfdf2"
Jul 13 19:20:36.169163 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:36.169133    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4wqcl_3693db18-1d4a-4c46-8cb9-75ba5c992a83/manager/0.log"
Jul 13 19:20:36.304949 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:36.304905    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zls45_5ce2c5df-34db-4e6f-861e-a080808a2530/manager/0.log"
Jul 13 19:20:36.452286 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:36.452243    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rcfd2_bb9b384f-4d45-4297-8e1c-94d06747ed00/server/0.log"
Jul 13 19:20:36.534188 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:36.534158    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hfcwh_80269e47-5ba4-4b0a-b925-06cbf69049bd/manager/0.log"
Jul 13 19:20:36.664120 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:36.664034    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4789cfbf-t92zq_76011d1d-de12-4769-b91f-e5600b77090a/manager/0.log"
Jul 13 19:20:36.682617 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:36.682594    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4789cfbf-t92zq_76011d1d-de12-4769-b91f-e5600b77090a/copy-manifests/0.log"
Jul 13 19:20:38.138890 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:38.138863    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6jt48_ba045cfc-ce28-452c-9bfc-f9d30a84d1fb/discovery/0.log"
Jul 13 19:20:38.178022 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:38.177994    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f8768797-p9r57_f0ab603f-cc81-4023-b68f-75c3bbdcaed7/kube-auth-proxy/0.log"
Jul 13 19:20:38.318116 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:38.318036    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-597d7847fb-55tbz_f5c0e7af-b62f-4dfc-810f-39223c968ce9/router/0.log"
Jul 13 19:20:39.699431 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:39.699392    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6jt48_ba045cfc-ce28-452c-9bfc-f9d30a84d1fb/discovery/0.log"
Jul 13 19:20:39.731141 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:39.731115    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f8768797-p9r57_f0ab603f-cc81-4023-b68f-75c3bbdcaed7/kube-auth-proxy/0.log"
Jul 13 19:20:39.870517 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:39.870437    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-597d7847fb-55tbz_f5c0e7af-b62f-4dfc-810f-39223c968ce9/router/0.log"
Jul 13 19:20:43.760953 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:43.760926    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-cndps_cc90ccc1-03cf-49db-bfd4-5458ec6e63a1/kube-storage-version-migrator-operator/1.log"
Jul 13 19:20:43.761797 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:43.761779    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-cndps_cc90ccc1-03cf-49db-bfd4-5458ec6e63a1/kube-storage-version-migrator-operator/0.log"
Jul 13 19:20:45.085918 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:45.085886    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fblxn_bf25eded-5569-4d4d-be04-4eb93300fcf2/kube-multus-additional-cni-plugins/0.log"
Jul 13 19:20:45.108257 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:45.108228    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fblxn_bf25eded-5569-4d4d-be04-4eb93300fcf2/egress-router-binary-copy/0.log"
Jul 13 19:20:45.129635 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:45.129610    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fblxn_bf25eded-5569-4d4d-be04-4eb93300fcf2/cni-plugins/0.log"
Jul 13 19:20:45.150320 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:45.150287    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fblxn_bf25eded-5569-4d4d-be04-4eb93300fcf2/bond-cni-plugin/0.log"
Jul 13 19:20:45.171259 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:45.171240    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fblxn_bf25eded-5569-4d4d-be04-4eb93300fcf2/routeoverride-cni/0.log"
Jul 13 19:20:45.192740 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:45.192720    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fblxn_bf25eded-5569-4d4d-be04-4eb93300fcf2/whereabouts-cni-bincopy/0.log"
Jul 13 19:20:45.213738 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:45.213705    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fblxn_bf25eded-5569-4d4d-be04-4eb93300fcf2/whereabouts-cni/0.log"
Jul 13 19:20:45.254244 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:45.254224    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c5db7_108324b4-322a-4ba1-9b6a-16e3ba7fea94/kube-multus/0.log"
Jul 13 19:20:45.408950 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:45.408875    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zvvkh_698c185e-1b6a-43a7-97fb-725ad56c0962/network-metrics-daemon/0.log"
Jul 13 19:20:45.433587 ip-10-0-139-206 kubenswrapper[2569]: I0713 19:20:45.433563    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zvvkh_698c185e-1b6a-43a7-97fb-725ad56c0962/kube-rbac-proxy/0.log"