Sep 11 11:51:30.267289 ip-10-0-136-222 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 11:51:30.689230 ip-10-0-136-222 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:51:30.689230 ip-10-0-136-222 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 11:51:30.689230 ip-10-0-136-222 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:51:30.689230 ip-10-0-136-222 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 11:51:30.689230 ip-10-0-136-222 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:51:30.690252 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.690165    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 11:51:30.694145 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694129    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:30.694145 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694145    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694150    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694153    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694156    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694159    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694162    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694165    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694168    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694171    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694173    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694177    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694180    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694182    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694185    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694196    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694199    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694201    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694204    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694206    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694209    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:30.694215 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694212    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694214    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694217    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694220    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694224    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694229    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694232    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694235    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694237    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694240    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694242    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694245    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694247    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694250    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694253    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694257    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694260    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694264    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694266    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:30.694801 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694269    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694272    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694274    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694277    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694279    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694283    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694288    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694291    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694294    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694296    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694299    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694301    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694304    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694306    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694309    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694313    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694315    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694319    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694321    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:30.695314 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694324    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694326    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694329    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694332    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694335    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694337    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694340    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694342    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694345    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694347    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694351    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694353    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694356    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694359    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694361    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694364    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694366    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694369    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694372    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694374    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:30.695785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694377    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694379    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694382    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694384    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694387    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694390    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694393    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694395    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694398    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694401    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694404    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694406    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694409    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694412    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694415    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694417    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694420    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694423    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694425    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.694428    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:30.696293 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695158    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695166    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695170    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695173    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695178    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695180    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695183    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695186    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695188    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695191    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695194    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695196    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695199    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695202    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695205    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695208    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695210    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695213    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695217    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:30.696785 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695221    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695225    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695228    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695231    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695234    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695237    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695240    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695243    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695246    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695248    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695251    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695254    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695257    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695259    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695262    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695265    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695267    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695270    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695272    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:30.697287 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695275    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695277    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695280    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695282    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695284    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695287    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695289    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695292    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695294    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695297    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695300    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695304    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695306    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695309    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695313    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695316    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695319    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695322    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695325    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695328    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:30.697743 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695331    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695334    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695336    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695339    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695343    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695345    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695348    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695350    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695353    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695355    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695358    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695362    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695365    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695368    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695371    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695373    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695376    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695378    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695381    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695384    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:30.698260 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695386    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695403    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695407    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695410    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695413    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695416    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695418    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695421    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695424    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695426    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695429    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695431    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695433    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695436    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695440    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695444    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695446    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695449    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695452    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695454    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:30.698746 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695457    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695820    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695825    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695828    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695831    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695833    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695836    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695839    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695841    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695844    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695847    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695850    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695853    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695855    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695858    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695863    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695866    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695869    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695871    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695873    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695876    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:30.699252 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695879    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695881    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695884    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695886    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695888    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695891    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695894    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695896    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695898    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695901    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695904    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695906    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695909    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695911    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695914    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695916    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695919    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695922    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695924    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695927    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:30.699784 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695929    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695932    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695934    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695937    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695940    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695942    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695946    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695951    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695954    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695972    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695975    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695978    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695981    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695984    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695987    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695989    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695992    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695995    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.695997    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:30.700280 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696000    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696003    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696006    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696009    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696011    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696014    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696017    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696020    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696023    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696025    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696029    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696031    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696034    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696037    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696040    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696042    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696045    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696048    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696050    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:30.700744 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696053    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696056    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696061    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696064    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696067    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696070    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696072    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696075    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696078    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696080    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696083    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696085    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696088    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696091    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696095    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696098    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696101    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696103    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696106    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:30.701230 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696109    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696111    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696183    2562 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696232    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696244    2562 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696248    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696252    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696256    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696260    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696264    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696268    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696271    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696274    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696278    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696281    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696285    2562 flags.go:64] FLAG: --cgroup-root=""
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696289    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696292    2562 flags.go:64] FLAG: --client-ca-file=""
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696295    2562 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696298    2562 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696302    2562 flags.go:64] FLAG: --cluster-domain=""
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696305    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696309    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696312    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 11:51:30.701678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696316    2562 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696319    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696322    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696326    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696329    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696333    2562 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696336    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696339    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696342    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696345    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696349    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696352    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696356    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696359    2562 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696362    2562 flags.go:64] FLAG: --enable-server="true"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696365    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696369    2562 flags.go:64] FLAG: --event-burst="100"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696372    2562 flags.go:64] FLAG: --event-qps="50"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696375    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696378    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696381    2562 flags.go:64] FLAG: --eviction-hard=""
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696385    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696387    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696392    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696394    2562 flags.go:64] FLAG: --eviction-soft=""
Sep 11 11:51:30.702281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696398    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696402    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696405    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696408    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696411    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696414    2562 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696417    2562 flags.go:64] FLAG: --feature-gates=""
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696421    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696424    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696427    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696430    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696434    2562 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696437    2562 flags.go:64] FLAG: --help="false"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696440    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-136-222.ec2.internal"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696443    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696446    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696449    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696452    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696456    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696460    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696463    2562 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696466    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696468    2562 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 11:51:30.702924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696471    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696474    2562 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696477    2562 flags.go:64] FLAG: --kube-reserved=""
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696480    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696483    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696486    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696489    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696493    2562 flags.go:64] FLAG: --lock-file=""
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696496    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696499    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696502    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696508    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696511    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696514    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696516    2562 flags.go:64] FLAG: --logging-format="text"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696519    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696523    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696525    2562 flags.go:64] FLAG: --manifest-url=""
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696529    2562 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696533    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696536    2562 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696539    2562 flags.go:64] FLAG: --max-pods="110"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696543    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696545    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696548    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 11:51:30.703574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696551    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696554    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696557    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696562    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696569    2562 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696572    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696575    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696578    2562 flags.go:64] FLAG: --pod-cidr=""
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696581    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696587    2562 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696590    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696593    2562 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696596    2562 flags.go:64] FLAG: --port="10250"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696599    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696604    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06bf1740e5c315789"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696608    2562 flags.go:64] FLAG: --qos-reserved=""
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696611    2562 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696614    2562 flags.go:64] FLAG: --register-node="true"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696617    2562 flags.go:64] FLAG: --register-with-taints=""
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696621    2562 flags.go:64] FLAG: --registry-burst="10"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696624    2562 flags.go:64] FLAG: --registry-qps="5"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696627    2562 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696630    2562 flags.go:64] FLAG: --reserved-memory=""
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696633    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696637    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 11:51:30.704319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696640    2562 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696643    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696646    2562 flags.go:64] FLAG: --runonce="false"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696649    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696652    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696655    2562 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696657    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696660    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696663    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696667    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696670    2562 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696675    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696678    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696681    2562 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696684    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696687    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696690    2562 flags.go:64] FLAG: --system-cgroups=""
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696693    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696698    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696701    2562 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696704    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696709    2562 flags.go:64] FLAG: --tls-min-version=""
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696712    2562 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696715    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696718    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 11:51:30.705330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696721    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696725    2562 flags.go:64] FLAG: --v="2"
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696729    2562 flags.go:64] FLAG: --version="false"
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696733    2562 flags.go:64] FLAG: --vmodule=""
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696737    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.696740    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696853    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696858    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696861    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696865    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696869    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696873    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696875    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696878    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696881    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696884    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696887    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696889    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696893    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696896    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:30.706033 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696898    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696901    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696904    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696907    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696910    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696912    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696915    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696917    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696921    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696923    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696926    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696929    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696931    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696934    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696938    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696940    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696943    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696946    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696949    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696951    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:30.706555 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696954    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696969    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696972    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696974    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696977    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696980    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.696982    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697000    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697004    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697007    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697012    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697015    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697018    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697021    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697024    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697027    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697030    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697033    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697036    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697038    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:30.707151 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697042    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697045    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697048    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697051    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697054    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697057    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697059    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697062    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697072    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697075    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697078    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697080    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697083    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697086    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697088    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697091    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697095    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697098    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697101    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:30.707626 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697104    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697107    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697110    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697115    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697118    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697120    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697123    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697126    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697129    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697131    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697134    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697137    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697139    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697142    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697145    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697148    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697151    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697153    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697156    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697158    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:30.708117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697161    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697164    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697167    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697170    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697172    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.697174    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.697909    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.707035    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.707052    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.707933    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.707943    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708026    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708032    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708036    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:30.708605 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708039    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708042    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708045    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708048    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708051    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708055    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708059    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708062    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708065    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708068    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708071    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708074    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708077    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708079    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708083    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708087    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708090    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708093    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708096    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:30.708974 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708099    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708102    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708105    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708108    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708110    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708113    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708116    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708119    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708122    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708125    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708128    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708130    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708133    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708136    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708138    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708141    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708144    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708146    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708149    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708152    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:30.709449 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708154    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708157    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708160    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708163    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708166    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708169    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708171    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708174    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708176    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708179    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708182    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708184    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708187    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708190    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708192    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708195    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708197    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708200    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708202    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708205    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:30.709938 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708208    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708210    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708213    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708216    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708218    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708221    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708223    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708226    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708229    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708231    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708234    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708237    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708240    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708242    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708245    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708247    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708250    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708255    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708258    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:30.710468 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708261    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708264    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708267    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708270    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708272    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708276    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708278    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708281    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708284    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708287    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708290    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708292    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708295    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708299    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708301    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708304    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708307    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:30.710937 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708309    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.708315    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708444    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708450    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708453    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708456    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708459    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708462    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708464    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708467    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708469    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708472    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708475    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708478    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:51:30.711413 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708481    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708484    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708487    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708489    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708492    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708495    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708497    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708500    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708503    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708506    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708508    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708511    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708514    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708516    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708518    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708521    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708524    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708526    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708529    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708531    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:51:30.711761 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708534    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708536    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708540    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708542    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708545    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708547    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708550    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708552    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708555    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708557    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708560    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708563    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708565    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708569    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708571    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708574    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708576    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708580    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708584    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:51:30.712298 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708586    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708589    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708592    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708595    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708598    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708600    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708603    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708606    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708609    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708612    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708615    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708618    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708620    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708623    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708625    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708628    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708630    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708633    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708636    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708638    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:51:30.712747 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708641    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708643    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708646    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708648    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708651    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708653    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708656    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708660    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708664    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708668    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708671    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708673    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708677    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708681    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708684    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708687    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708690    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708693    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708695    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:51:30.713236 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708698    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708701    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708703    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708706    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708708    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708711    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708714    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708716    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:30.708719    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.708724    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.708848    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 11:51:30.713717 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.713049    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 11:51:30.714121 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.714108    2562 server.go:1019] "Starting client certificate rotation"
Sep 11 11:51:30.714245 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.714226    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 11:51:30.714301 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.714278    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 11:51:30.716971 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.716942    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 11:51:30.722462 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.722435    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 11:51:30.729844 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.729826    2562 log.go:25] "Validated CRI v1 runtime API"
Sep 11 11:51:30.733992 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.733977    2562 log.go:25] "Validated CRI v1 image API"
Sep 11 11:51:30.738347 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.738320    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 11:51:30.740921 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.740901    2562 fs.go:135] Filesystem UUIDs: map[09615ffd-cb00-4eea-ad63-15314e510993:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fa6de7b4-3a9b-4032-96c3-cc885b31c1ae:/dev/nvme0n1p4]
Sep 11 11:51:30.741023 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.740920    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 11:51:30.746099 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.745985    2562 manager.go:216] Machine: {Timestamp:2026-09-11 11:51:30.745477466 +0000 UTC m=+0.373542988 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099907 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25600da89a7cba1cf2881895ae4851 SystemUUID:ec25600d-a89a-7cba-1cf2-881895ae4851 BootID:55e0d6c1-1b92-4abc-abde-330d1b1abc1e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4c:ad:55:a5:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4c:ad:55:a5:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:76:2b:79:9f:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 11:51:30.746099 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.746091    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 11:51:30.746274 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.746204    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 11:51:30.746751 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.746735    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 11:51:30.749210 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.749182    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 11:51:30.749352 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.749214    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-222.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 11:51:30.750094 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.750084    2562 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 11:51:30.750138 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.750096    2562 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 11:51:30.750138 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.750105    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 11:51:30.750138 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.750119    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 11:51:30.751396 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.751386    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 11:51:30.756143 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.756131    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 11:51:30.756271 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.756261    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 11:51:30.756318 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.756292    2562 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 11:51:30.756318 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.756309    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 11:51:30.756405 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.756327    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 11:51:30.756405 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.756337    2562 kubelet.go:398] "Adding apiserver pod source"
Sep 11 11:51:30.756405 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.756347    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 11:51:30.756750 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.756736    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 11:51:30.756805 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.756762    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 11:51:30.759227 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.759209    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 11:51:30.762158 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.762141    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 11:51:30.762233 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.762180    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 11:51:30.764248 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764234    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 11:51:30.764325 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764257    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 11:51:30.764325 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764271    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 11:51:30.764325 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764282    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 11:51:30.764325 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764302    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 11:51:30.764325 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764313    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 11:51:30.764498 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764337    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 11:51:30.764498 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764347    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 11:51:30.764498 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764358    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 11:51:30.764498 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764369    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 11:51:30.764498 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764373    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rgxm6"
Sep 11 11:51:30.764498 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764387    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 11:51:30.764498 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.764499    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 11:51:30.766369 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.766358    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 11:51:30.766405 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.766371    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 11:51:30.766470 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.766442    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 11:51:30.766596 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.766581    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-222.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 11:51:30.770359 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.770346    2562 server.go:1268] "Started kubelet"
Sep 11 11:51:30.771038 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.771007    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 11:51:30.771334 ip-10-0-136-222 systemd[1]: Started Kubernetes Kubelet.
Sep 11 11:51:30.771549 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.771290    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 11:51:30.771549 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.771380    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 11:51:30.772464 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.772445    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rgxm6"
Sep 11 11:51:30.773004 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.772985    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 11:51:30.773925 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.773909    2562 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 11:51:30.776914 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.776897    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 11:51:30.778256 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.778235    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-222.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 11:51:30.779370 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.778214    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-222.ec2.internal.18d441f9413f3d71  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-222.ec2.internal,UID:ip-10-0-136-222.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-222.ec2.internal,},FirstTimestamp:2026-09-11 11:51:30.770255217 +0000 UTC m=+0.398320743,LastTimestamp:2026-09-11 11:51:30.770255217 +0000 UTC m=+0.398320743,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-222.ec2.internal,}"
Sep 11 11:51:30.782299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.782282    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 11:51:30.783197 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.783158    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 11:51:30.783500 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.783451    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:30.784044 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.784016    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 11:51:30.784044 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.784020    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 11:51:30.784044 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.784044    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 11:51:30.784228 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.784170    2562 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 11:51:30.784228 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.784178    2562 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 11:51:30.785061 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.785041    2562 factory.go:55] Registering systemd factory
Sep 11 11:51:30.785061 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.785064    2562 factory.go:223] Registration of the systemd container factory successfully
Sep 11 11:51:30.785410 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.785395    2562 factory.go:153] Registering CRI-O factory
Sep 11 11:51:30.785410 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.785412    2562 factory.go:223] Registration of the crio container factory successfully
Sep 11 11:51:30.785540 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.785460    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-222.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 11:51:30.785540 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.785473    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 11:51:30.785540 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.785502    2562 factory.go:103] Registering Raw factory
Sep 11 11:51:30.785540 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.785519    2562 manager.go:1196] Started watching for new ooms in manager
Sep 11 11:51:30.785722 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.785547    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 11:51:30.786031 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.786011    2562 manager.go:349] Starting recovery of all containers
Sep 11 11:51:30.798092 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.797919    2562 manager.go:354] Recovery completed
Sep 11 11:51:30.802548 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.802534    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:51:30.804528 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.804507    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:51:30.804597 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.804542    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:51:30.804597 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.804557    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:51:30.805210 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.805192    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 11:51:30.805210 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.805209    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 11:51:30.805360 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.805211    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:51:30.805360 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.805226    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 11:51:30.806223 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.806204    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:51:30.806299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.806239    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:51:30.806299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.806253    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:51:30.807338 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.807322    2562 policy_none.go:49] "None policy: Start"
Sep 11 11:51:30.807338 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.807340    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 11:51:30.807472 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.807353    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 11:51:30.809347 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.809333    2562 policy_none.go:47] "Start"
Sep 11 11:51:30.853568 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.853551    2562 manager.go:337] "Starting Device Plugin manager"
Sep 11 11:51:30.853670 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.853587    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 11:51:30.853670 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.853598    2562 server.go:87] "Starting device plugin registration server"
Sep 11 11:51:30.853983 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.853954    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 11:51:30.854078 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.853990    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 11:51:30.854078 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.854073    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 11:51:30.854175 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.854145    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 11:51:30.854175 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.854153    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 11:51:30.854676 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.854655    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 11:51:30.854757 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.854701    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:30.889539 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.889486    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 11:51:30.890734 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.890721    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 11:51:30.890818 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.890780    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 11:51:30.890818 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.890809    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 11:51:30.890934 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.890919    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 11:51:30.893163 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.893143    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:30.955168 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.955079    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:51:30.956165 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.956147    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:51:30.956245 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.956183    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:51:30.956245 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.956198    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:51:30.956245 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.956223    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-222.ec2.internal"
Sep 11 11:51:30.962390 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.962375    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-222.ec2.internal"
Sep 11 11:51:30.962437 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.962399    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-222.ec2.internal\": node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:30.975597 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:30.975572    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:30.991514 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.991491    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal"]
Sep 11 11:51:30.991573 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.991555    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:51:30.993090 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.993075    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:51:30.993152 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.993104    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:51:30.993152 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.993115    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:51:30.995687 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.995673    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:51:30.995856 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.995841    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:30.995929 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.995882    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:51:30.996395 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.996376    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:51:30.996484 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.996401    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:51:30.996484 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.996411    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:51:30.996484 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.996456    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:51:30.996847 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.996803    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:51:30.996847 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.996832    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:51:30.999228 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.999210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:30.999293 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.999242    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:51:30.999899 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.999885    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:51:30.999978 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.999917    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:51:30.999978 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:30.999932    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:51:31.022652 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:31.022630    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-222.ec2.internal\" not found" node="ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.026952 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:31.026936    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-222.ec2.internal\" not found" node="ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.076132 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:31.076102    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:31.085386 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.085365    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a890768e232d08590a8e99d453224f11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal\" (UID: \"a890768e232d08590a8e99d453224f11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.085488 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.085392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a890768e232d08590a8e99d453224f11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal\" (UID: \"a890768e232d08590a8e99d453224f11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.085488 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.085412    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0b320866b4a982a72db4f26f5aaaff86-config\") pod \"kube-apiserver-proxy-ip-10-0-136-222.ec2.internal\" (UID: \"0b320866b4a982a72db4f26f5aaaff86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.176821 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:31.176783    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:31.186149 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.186128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a890768e232d08590a8e99d453224f11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal\" (UID: \"a890768e232d08590a8e99d453224f11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.186209 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.186167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a890768e232d08590a8e99d453224f11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal\" (UID: \"a890768e232d08590a8e99d453224f11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.186209 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.186199    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0b320866b4a982a72db4f26f5aaaff86-config\") pod \"kube-apiserver-proxy-ip-10-0-136-222.ec2.internal\" (UID: \"0b320866b4a982a72db4f26f5aaaff86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.186209 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.186189    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a890768e232d08590a8e99d453224f11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal\" (UID: \"a890768e232d08590a8e99d453224f11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.186318 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.186227    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a890768e232d08590a8e99d453224f11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal\" (UID: \"a890768e232d08590a8e99d453224f11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.186318 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.186271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0b320866b4a982a72db4f26f5aaaff86-config\") pod \"kube-apiserver-proxy-ip-10-0-136-222.ec2.internal\" (UID: \"0b320866b4a982a72db4f26f5aaaff86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.277613 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:31.277536    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:31.325110 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.325083    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.329897 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:31.329865    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:31.378440 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:31.378410    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:32.606404 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:32.606330    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:32.706930 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:32.706835    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:32.733642 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:32.733617    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:32.807672 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:32.807644    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:32.841101 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:32.841077    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 11:51:32.841514 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:32.841207    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 11:51:32.841514 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:32.841236    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 11:51:32.908124 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:32.908091    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:32.909218 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:32.909181    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 11:46:31 +0000 UTC" deadline="2028-03-21 03:49:45.754971212 +0000 UTC"
Sep 11 11:51:32.909218 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:32.909217    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13359h58m12.84575674s"
Sep 11 11:51:33.008432 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:33.008354    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-222.ec2.internal\" not found"
Sep 11 11:51:33.037988 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.037960    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:33.111701 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.111675    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:33.122562 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.122544    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 11:51:33.123881 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.123867    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal"
Sep 11 11:51:33.129880 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.129864    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 11:51:33.204117 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:33.204079    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda890768e232d08590a8e99d453224f11.slice/crio-dc69726a6bf4b19c65826437f851ac501bd60e971a5ef35a76d46fa19bfec11d WatchSource:0}: Error finding container dc69726a6bf4b19c65826437f851ac501bd60e971a5ef35a76d46fa19bfec11d: Status 404 returned error can't find the container with id dc69726a6bf4b19c65826437f851ac501bd60e971a5ef35a76d46fa19bfec11d
Sep 11 11:51:33.204448 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:33.204426    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b320866b4a982a72db4f26f5aaaff86.slice/crio-1e796e8e854add2d5fe6ace21a77562bed378a2fc45dd570e251c4f2eb6656a8 WatchSource:0}: Error finding container 1e796e8e854add2d5fe6ace21a77562bed378a2fc45dd570e251c4f2eb6656a8: Status 404 returned error can't find the container with id 1e796e8e854add2d5fe6ace21a77562bed378a2fc45dd570e251c4f2eb6656a8
Sep 11 11:51:33.209389 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.209375    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 11:51:33.368756 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.368679    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:33.884745 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.884706    2562 apiserver.go:52] "Watching apiserver"
Sep 11 11:51:33.889907 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.889872    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 11:51:33.891192 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.891147    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-6f2ph","kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal","openshift-cluster-node-tuning-operator/tuned-rfckf","openshift-dns/node-resolver-xtfmj","openshift-image-registry/node-ca-lv7nr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal","openshift-multus/multus-additional-cni-plugins-nkpgh","openshift-multus/network-metrics-daemon-w98wm","kube-system/global-pull-secret-syncer-fqcqr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp","openshift-multus/multus-pxcxn","openshift-network-diagnostics/network-check-target-t8djg","openshift-network-operator/iptables-alerter-qnd89","openshift-ovn-kubernetes/ovnkube-node-f8cvg"]
Sep 11 11:51:33.894229 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.894184    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:33.898697 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.898672    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 11:51:33.900252 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.899004    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 11:51:33.900252 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.899203    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 11:51:33.900252 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.899424    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 11:51:33.900632 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.900470    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qnth2\""
Sep 11 11:51:33.902046 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.901401    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:33.907873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.902904    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 11:51:33.907873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.903763    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 11:51:33.907873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.903935    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:51:33.907873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.904095    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 11:51:33.907873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.904623    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.907873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.906437    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lgppv\""
Sep 11 11:51:33.907873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.906749    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:33.907873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.907347    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 11:51:33.907873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.907492    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:51:33.907873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.907426    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-557wt\""
Sep 11 11:51:33.909961 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.908661    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:33.909961 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.908992    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 11:51:33.909961 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.909025    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 11:51:33.909961 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.909386    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vmf5j\""
Sep 11 11:51:33.910787 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.910676    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 11:51:33.911245 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.911226    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 11:51:33.912097 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.911377    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 11:51:33.913122 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.912294    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4568k\""
Sep 11 11:51:33.913122 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.912443    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 11:51:33.914686 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.914666    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.916185 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.916163    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:33.916274 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:33.916234    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:33.916507 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.916487    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 11:51:33.917135 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.917116    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 11:51:33.917492 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.917227    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c6kvj\""
Sep 11 11:51:33.917633 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.917612    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 11:51:33.918026 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.917993    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 11:51:33.918554 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.918349    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 11:51:33.918554 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.918435    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 11:51:33.918811 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.918782    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:33.920921 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.920805    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 11:51:33.921289 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.921272    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 11:51:33.921625 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.921608    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v4ls5\""
Sep 11 11:51:33.921974 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.921680    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:33.921974 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:33.921920    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:33.922632 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1264750e-2181-40ee-8953-fc6c444b9bd5-iptables-alerter-script\") pod \"iptables-alerter-qnd89\" (UID: \"1264750e-2181-40ee-8953-fc6c444b9bd5\") " pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:33.922632 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcm52\" (UniqueName: \"kubernetes.io/projected/1264750e-2181-40ee-8953-fc6c444b9bd5-kube-api-access-zcm52\") pod \"iptables-alerter-qnd89\" (UID: \"1264750e-2181-40ee-8953-fc6c444b9bd5\") " pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:33.922632 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922574    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-kubernetes\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.922862 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922633    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0d8a49a-8299-402c-83b2-4de5f94b5552-tmp\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.922862 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922665    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5vgl\" (UniqueName: \"kubernetes.io/projected/86e098b8-e73f-43c1-bfe3-242bd950b4bd-kube-api-access-r5vgl\") pod \"node-resolver-xtfmj\" (UID: \"86e098b8-e73f-43c1-bfe3-242bd950b4bd\") " pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:33.922862 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922687    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm7xh\" (UniqueName: \"kubernetes.io/projected/1e028310-ec98-4645-ad5c-48db2103260d-kube-api-access-pm7xh\") pod \"node-ca-lv7nr\" (UID: \"1e028310-ec98-4645-ad5c-48db2103260d\") " pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:33.922862 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922716    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-var-lib-openvswitch\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.922862 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922754    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-node-log\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.922862 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922782    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c05f73e-008f-497c-9e03-6b411a683a03-cni-binary-copy\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:33.922862 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922818    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k98v\" (UniqueName: \"kubernetes.io/projected/7c05f73e-008f-497c-9e03-6b411a683a03-kube-api-access-4k98v\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:33.922862 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922852    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-var-lib-kubelet\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922895    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dskz\" (UniqueName: \"kubernetes.io/projected/e0d8a49a-8299-402c-83b2-4de5f94b5552-kube-api-access-6dskz\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922925    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-cni-netd\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922952    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.922980    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5d25bbd-8989-4fae-912f-329c7904574e-ovnkube-config\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923004    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqq4g\" (UniqueName: \"kubernetes.io/projected/e5d25bbd-8989-4fae-912f-329c7904574e-kube-api-access-xqq4g\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923047    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c05f73e-008f-497c-9e03-6b411a683a03-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923075    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-sysconfig\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923097    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-lib-modules\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86e098b8-e73f-43c1-bfe3-242bd950b4bd-hosts-file\") pod \"node-resolver-xtfmj\" (UID: \"86e098b8-e73f-43c1-bfe3-242bd950b4bd\") " pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923163    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5d25bbd-8989-4fae-912f-329c7904574e-ovn-node-metrics-cert\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1264750e-2181-40ee-8953-fc6c444b9bd5-host-slash\") pod \"iptables-alerter-qnd89\" (UID: \"1264750e-2181-40ee-8953-fc6c444b9bd5\") " pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923210    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-host\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923233    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-systemd-units\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923254    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-run-netns\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923283    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-etc-openvswitch\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923297    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-log-socket\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5d25bbd-8989-4fae-912f-329c7904574e-env-overrides\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923385    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-sysctl-conf\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923422    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-sys\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923445    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-tuned\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923470    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-run-ovn-kubernetes\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c05f73e-008f-497c-9e03-6b411a683a03-system-cni-dir\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923534    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-sysctl-d\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-run-openvswitch\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923578    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-modprobe-d\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923647    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-systemd\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923668    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-run\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86e098b8-e73f-43c1-bfe3-242bd950b4bd-tmp-dir\") pod \"node-resolver-xtfmj\" (UID: \"86e098b8-e73f-43c1-bfe3-242bd950b4bd\") " pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923722    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e028310-ec98-4645-ad5c-48db2103260d-serviceca\") pod \"node-ca-lv7nr\" (UID: \"1e028310-ec98-4645-ad5c-48db2103260d\") " pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923744    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-kubelet\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.923980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923770    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-slash\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.924848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923804    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-run-ovn\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.924848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c05f73e-008f-497c-9e03-6b411a683a03-cnibin\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:33.924848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923874    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c05f73e-008f-497c-9e03-6b411a683a03-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:33.924848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923898    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c05f73e-008f-497c-9e03-6b411a683a03-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:33.924848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e028310-ec98-4645-ad5c-48db2103260d-host\") pod \"node-ca-lv7nr\" (UID: \"1e028310-ec98-4645-ad5c-48db2103260d\") " pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:33.924848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923962    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-run-systemd\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.924848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.923983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-cni-bin\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.924848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.924006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5d25bbd-8989-4fae-912f-329c7904574e-ovnkube-script-lib\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:33.924848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.924606    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 11:51:33.925336 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.925043    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 11:51:33.925385 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.925368    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 11:51:33.925457 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.925395    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 11:51:33.925636 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.925619    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-prp2g\""
Sep 11 11:51:33.925760 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.925735    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:33.928079 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.927905    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 11:51:33.928079 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.928043    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p4pzz\""
Sep 11 11:51:33.928238 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.928173    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:33.928297 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:33.928230    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:33.948255 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.948233    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-775pj"
Sep 11 11:51:33.953804 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:33.953751    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-775pj"
Sep 11 11:51:34.013510 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.013482    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-registration-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-627ff\" (UniqueName: \"kubernetes.io/projected/f2ce1114-8dd3-4330-a889-4d826243cb38-kube-api-access-627ff\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024256    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-run-netns\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-sys-fs\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-conf-dir\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024386    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e693b077-566f-41ce-b702-f9ea0084ed60-kubelet-config\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024414    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024440    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b4vb\" (UniqueName: \"kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb\") pod \"network-check-target-t8djg\" (UID: \"1a71dff7-c64a-402f-ac3f-a0c2bb078721\") " pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5d25bbd-8989-4fae-912f-329c7904574e-ovnkube-script-lib\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-modprobe-d\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024549    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-systemd\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-run\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024593    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-run-ovn\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024615    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-var-lib-cni-multus\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c05f73e-008f-497c-9e03-6b411a683a03-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c05f73e-008f-497c-9e03-6b411a683a03-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.024658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024675    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e028310-ec98-4645-ad5c-48db2103260d-host\") pod \"node-ca-lv7nr\" (UID: \"1e028310-ec98-4645-ad5c-48db2103260d\") " pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024697    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-system-cni-dir\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-modprobe-d\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024719    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-cni-dir\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024741    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-run-k8s-cni-cncf-io\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-daemon-config\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024782    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e693b077-566f-41ce-b702-f9ea0084ed60-dbus\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-kubernetes\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024826    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0d8a49a-8299-402c-83b2-4de5f94b5552-tmp\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-var-lib-openvswitch\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-node-log\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024931    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5d25bbd-8989-4fae-912f-329c7904574e-ovnkube-config\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024942    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c05f73e-008f-497c-9e03-6b411a683a03-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.024957    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c05f73e-008f-497c-9e03-6b411a683a03-cni-binary-copy\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-run\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025092    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-var-lib-kubelet\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.025299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025116    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqq4g\" (UniqueName: \"kubernetes.io/projected/e5d25bbd-8989-4fae-912f-329c7904574e-kube-api-access-xqq4g\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025143    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c05f73e-008f-497c-9e03-6b411a683a03-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-sysconfig\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025240    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86e098b8-e73f-43c1-bfe3-242bd950b4bd-hosts-file\") pod \"node-resolver-xtfmj\" (UID: \"86e098b8-e73f-43c1-bfe3-242bd950b4bd\") " pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025263    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-device-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-etc-selinux\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025337    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-var-lib-cni-bin\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025341    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-systemd\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1264750e-2181-40ee-8953-fc6c444b9bd5-host-slash\") pod \"iptables-alerter-qnd89\" (UID: \"1264750e-2181-40ee-8953-fc6c444b9bd5\") " pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025412    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025426    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5d25bbd-8989-4fae-912f-329c7904574e-ovnkube-script-lib\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025451    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e028310-ec98-4645-ad5c-48db2103260d-host\") pod \"node-ca-lv7nr\" (UID: \"1e028310-ec98-4645-ad5c-48db2103260d\") " pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-sysconfig\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86e098b8-e73f-43c1-bfe3-242bd950b4bd-hosts-file\") pod \"node-resolver-xtfmj\" (UID: \"86e098b8-e73f-43c1-bfe3-242bd950b4bd\") " pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025562    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c05f73e-008f-497c-9e03-6b411a683a03-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-node-log\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.026034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-kubernetes\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025643    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025663    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-run-ovn\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025677    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-var-lib-openvswitch\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025714    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-var-lib-kubelet\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025751    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-host\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025785    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-etc-openvswitch\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-log-socket\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025866    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-cni-binary-copy\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025896    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-sysctl-conf\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-sys\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.025958    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c05f73e-008f-497c-9e03-6b411a683a03-cni-binary-copy\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026006    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-tuned\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026035    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-run-ovn-kubernetes\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026066    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-socket-dir-parent\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c05f73e-008f-497c-9e03-6b411a683a03-system-cni-dir\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-log-socket\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.026705 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5d25bbd-8989-4fae-912f-329c7904574e-ovnkube-config\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-sysctl-d\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-run-openvswitch\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-etc-openvswitch\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1264750e-2181-40ee-8953-fc6c444b9bd5-host-slash\") pod \"iptables-alerter-qnd89\" (UID: \"1264750e-2181-40ee-8953-fc6c444b9bd5\") " pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026242    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c05f73e-008f-497c-9e03-6b411a683a03-system-cni-dir\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026254    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-host\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026283    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-run-openvswitch\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026213    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-run-ovn-kubernetes\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-sys\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026452    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-sysctl-d\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026437    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c05f73e-008f-497c-9e03-6b411a683a03-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026536    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd-konnectivity-ca\") pod \"konnectivity-agent-6f2ph\" (UID: \"a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd\") " pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026512    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal" event={"ID":"a890768e232d08590a8e99d453224f11","Type":"ContainerStarted","Data":"dc69726a6bf4b19c65826437f851ac501bd60e971a5ef35a76d46fa19bfec11d"}
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026560    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-sysctl-conf\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-socket-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026613    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-cni-bin\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.027496 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026638    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86e098b8-e73f-43c1-bfe3-242bd950b4bd-tmp-dir\") pod \"node-resolver-xtfmj\" (UID: \"86e098b8-e73f-43c1-bfe3-242bd950b4bd\") " pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026665    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e028310-ec98-4645-ad5c-48db2103260d-serviceca\") pod \"node-ca-lv7nr\" (UID: \"1e028310-ec98-4645-ad5c-48db2103260d\") " pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026681    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-cni-bin\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-kubelet\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026734    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-slash\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026754    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-kubelet\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-slash\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.026978    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-etc-kubernetes\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027019    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c05f73e-008f-497c-9e03-6b411a683a03-cnibin\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027056    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-run-systemd\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027073    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c05f73e-008f-497c-9e03-6b411a683a03-cnibin\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86e098b8-e73f-43c1-bfe3-242bd950b4bd-tmp-dir\") pod \"node-resolver-xtfmj\" (UID: \"86e098b8-e73f-43c1-bfe3-242bd950b4bd\") " pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-run-multus-certs\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1264750e-2181-40ee-8953-fc6c444b9bd5-iptables-alerter-script\") pod \"iptables-alerter-qnd89\" (UID: \"1264750e-2181-40ee-8953-fc6c444b9bd5\") " pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027126    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-run-systemd\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027158    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcm52\" (UniqueName: \"kubernetes.io/projected/1264750e-2181-40ee-8953-fc6c444b9bd5-kube-api-access-zcm52\") pod \"iptables-alerter-qnd89\" (UID: \"1264750e-2181-40ee-8953-fc6c444b9bd5\") " pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027160    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e028310-ec98-4645-ad5c-48db2103260d-serviceca\") pod \"node-ca-lv7nr\" (UID: \"1e028310-ec98-4645-ad5c-48db2103260d\") " pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027400    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5vgl\" (UniqueName: \"kubernetes.io/projected/86e098b8-e73f-43c1-bfe3-242bd950b4bd-kube-api-access-r5vgl\") pod \"node-resolver-xtfmj\" (UID: \"86e098b8-e73f-43c1-bfe3-242bd950b4bd\") " pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:34.028298 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm7xh\" (UniqueName: \"kubernetes.io/projected/1e028310-ec98-4645-ad5c-48db2103260d-kube-api-access-pm7xh\") pod \"node-ca-lv7nr\" (UID: \"1e028310-ec98-4645-ad5c-48db2103260d\") " pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-cni-netd\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z9pb\" (UniqueName: \"kubernetes.io/projected/142371cb-79c5-44b6-9e7d-9c01359b29f4-kube-api-access-9z9pb\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027504    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-cnibin\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-var-lib-kubelet\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027549    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-cni-netd\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k98v\" (UniqueName: \"kubernetes.io/projected/7c05f73e-008f-497c-9e03-6b411a683a03-kube-api-access-4k98v\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027585    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dskz\" (UniqueName: \"kubernetes.io/projected/e0d8a49a-8299-402c-83b2-4de5f94b5552-kube-api-access-6dskz\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027612    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-hostroot\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027635    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv5q4\" (UniqueName: \"kubernetes.io/projected/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-kube-api-access-zv5q4\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027685    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-lib-modules\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5d25bbd-8989-4fae-912f-329c7904574e-ovn-node-metrics-cert\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027710    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1264750e-2181-40ee-8953-fc6c444b9bd5-iptables-alerter-script\") pod \"iptables-alerter-qnd89\" (UID: \"1264750e-2181-40ee-8953-fc6c444b9bd5\") " pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-systemd-units\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-run-netns\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5d25bbd-8989-4fae-912f-329c7904574e-env-overrides\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.029154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027802    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-systemd-units\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.029932 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027817    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd-agent-certs\") pod \"konnectivity-agent-6f2ph\" (UID: \"a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd\") " pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:34.029932 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027907    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0d8a49a-8299-402c-83b2-4de5f94b5552-lib-modules\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.029932 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.027910    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5d25bbd-8989-4fae-912f-329c7904574e-host-run-netns\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.029932 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.028135    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal" event={"ID":"0b320866b4a982a72db4f26f5aaaff86","Type":"ContainerStarted","Data":"1e796e8e854add2d5fe6ace21a77562bed378a2fc45dd570e251c4f2eb6656a8"}
Sep 11 11:51:34.029932 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.028182    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5d25bbd-8989-4fae-912f-329c7904574e-env-overrides\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.029932 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.029876    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e0d8a49a-8299-402c-83b2-4de5f94b5552-etc-tuned\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.030469 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.030409    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0d8a49a-8299-402c-83b2-4de5f94b5552-tmp\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.033333 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.033150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5d25bbd-8989-4fae-912f-329c7904574e-ovn-node-metrics-cert\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.033720 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.033695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqq4g\" (UniqueName: \"kubernetes.io/projected/e5d25bbd-8989-4fae-912f-329c7904574e-kube-api-access-xqq4g\") pod \"ovnkube-node-f8cvg\" (UID: \"e5d25bbd-8989-4fae-912f-329c7904574e\") " pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.035246 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.035219    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5vgl\" (UniqueName: \"kubernetes.io/projected/86e098b8-e73f-43c1-bfe3-242bd950b4bd-kube-api-access-r5vgl\") pod \"node-resolver-xtfmj\" (UID: \"86e098b8-e73f-43c1-bfe3-242bd950b4bd\") " pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:34.036650 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.036564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k98v\" (UniqueName: \"kubernetes.io/projected/7c05f73e-008f-497c-9e03-6b411a683a03-kube-api-access-4k98v\") pod \"multus-additional-cni-plugins-nkpgh\" (UID: \"7c05f73e-008f-497c-9e03-6b411a683a03\") " pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.037403 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.037238    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm7xh\" (UniqueName: \"kubernetes.io/projected/1e028310-ec98-4645-ad5c-48db2103260d-kube-api-access-pm7xh\") pod \"node-ca-lv7nr\" (UID: \"1e028310-ec98-4645-ad5c-48db2103260d\") " pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:34.037403 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.037268    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dskz\" (UniqueName: \"kubernetes.io/projected/e0d8a49a-8299-402c-83b2-4de5f94b5552-kube-api-access-6dskz\") pod \"tuned-rfckf\" (UID: \"e0d8a49a-8299-402c-83b2-4de5f94b5552\") " pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.038977 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.038953    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcm52\" (UniqueName: \"kubernetes.io/projected/1264750e-2181-40ee-8953-fc6c444b9bd5-kube-api-access-zcm52\") pod \"iptables-alerter-qnd89\" (UID: \"1264750e-2181-40ee-8953-fc6c444b9bd5\") " pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:34.128667 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.128667 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-device-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128702    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-etc-selinux\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-var-lib-cni-bin\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-cni-binary-copy\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128780    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-socket-dir-parent\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128811    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd-konnectivity-ca\") pod \"konnectivity-agent-6f2ph\" (UID: \"a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd\") " pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128835    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-socket-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-device-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128851    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-var-lib-cni-bin\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128864    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-etc-kubernetes\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128904    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-etc-kubernetes\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-run-multus-certs\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.128928 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128934    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-run-multus-certs\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128940    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z9pb\" (UniqueName: \"kubernetes.io/projected/142371cb-79c5-44b6-9e7d-9c01359b29f4-kube-api-access-9z9pb\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-etc-selinux\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.128979    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-cnibin\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129009    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-var-lib-kubelet\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129034    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-hostroot\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129056    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv5q4\" (UniqueName: \"kubernetes.io/projected/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-kube-api-access-zv5q4\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129082    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129145    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd-agent-certs\") pod \"konnectivity-agent-6f2ph\" (UID: \"a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd\") " pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-registration-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-627ff\" (UniqueName: \"kubernetes.io/projected/f2ce1114-8dd3-4330-a889-4d826243cb38-kube-api-access-627ff\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-run-netns\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129248    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-sys-fs\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129273    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-conf-dir\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129286    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-var-lib-kubelet\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-socket-dir-parent\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129300    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e693b077-566f-41ce-b702-f9ea0084ed60-kubelet-config\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:34.129550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129362    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-hostroot\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4vb\" (UniqueName: \"kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb\") pod \"network-check-target-t8djg\" (UID: \"1a71dff7-c64a-402f-ac3f-a0c2bb078721\") " pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129372    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-cnibin\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-var-lib-cni-multus\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129432    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-sys-fs\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-system-cni-dir\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-cni-dir\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.129473    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129509    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-run-k8s-cni-cncf-io\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-daemon-config\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e693b077-566f-41ce-b702-f9ea0084ed60-dbus\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129572    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-var-lib-cni-multus\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129628    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-cni-dir\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-socket-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.129658    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret podName:e693b077-566f-41ce-b702-f9ea0084ed60 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:34.629632393 +0000 UTC m=+3.130352656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret") pod "global-pull-secret-syncer-fqcqr" (UID: "e693b077-566f-41ce-b702-f9ea0084ed60") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129534    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-cni-binary-copy\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129709    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-run-k8s-cni-cncf-io\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.130406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-conf-dir\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.131118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e693b077-566f-41ce-b702-f9ea0084ed60-kubelet-config\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:34.131118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129839    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-host-run-netns\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.131118 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.129890    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:34.131118 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.129947    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs podName:142371cb-79c5-44b6-9e7d-9c01359b29f4 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:34.629929929 +0000 UTC m=+3.130650176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs") pod "network-metrics-daemon-w98wm" (UID: "142371cb-79c5-44b6-9e7d-9c01359b29f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:34.131118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.129986    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e693b077-566f-41ce-b702-f9ea0084ed60-dbus\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:34.131118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.130031    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2ce1114-8dd3-4330-a889-4d826243cb38-registration-dir\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.131118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.130029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-system-cni-dir\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.131118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.130097    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd-konnectivity-ca\") pod \"konnectivity-agent-6f2ph\" (UID: \"a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd\") " pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:34.131118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.130327    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-multus-daemon-config\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.132546 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.132521    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd-agent-certs\") pod \"konnectivity-agent-6f2ph\" (UID: \"a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd\") " pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:34.137340 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.137256    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:34.137340 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.137292    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:34.137340 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.137337    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4b4vb for pod openshift-network-diagnostics/network-check-target-t8djg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:34.137553 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.137458    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb podName:1a71dff7-c64a-402f-ac3f-a0c2bb078721 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:34.637432052 +0000 UTC m=+3.138152317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4vb" (UniqueName: "kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb") pod "network-check-target-t8djg" (UID: "1a71dff7-c64a-402f-ac3f-a0c2bb078721") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:34.138486 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.138461    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-627ff\" (UniqueName: \"kubernetes.io/projected/f2ce1114-8dd3-4330-a889-4d826243cb38-kube-api-access-627ff\") pod \"aws-ebs-csi-driver-node-rqjmp\" (UID: \"f2ce1114-8dd3-4330-a889-4d826243cb38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.141085 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.141061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv5q4\" (UniqueName: \"kubernetes.io/projected/2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6-kube-api-access-zv5q4\") pod \"multus-pxcxn\" (UID: \"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6\") " pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.141216 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.141193    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z9pb\" (UniqueName: \"kubernetes.io/projected/142371cb-79c5-44b6-9e7d-9c01359b29f4-kube-api-access-9z9pb\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:34.209198 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.209166    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:34.217948 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.217912    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nkpgh"
Sep 11 11:51:34.226921 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.226898    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qnd89"
Sep 11 11:51:34.237850 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.237816    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rfckf"
Sep 11 11:51:34.244572 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.244548    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xtfmj"
Sep 11 11:51:34.254266 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.254244    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lv7nr"
Sep 11 11:51:34.260079 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.260058    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:51:34.269779 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.269756    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:34.289475 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.289452    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp"
Sep 11 11:51:34.295204 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.295185    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pxcxn"
Sep 11 11:51:34.632715 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.632614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:34.632715 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.632668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:34.632948 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.632784    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:34.632948 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.632847    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs podName:142371cb-79c5-44b6-9e7d-9c01359b29f4 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:35.632828677 +0000 UTC m=+4.133548943 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs") pod "network-metrics-daemon-w98wm" (UID: "142371cb-79c5-44b6-9e7d-9c01359b29f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:34.632948 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.632787    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:34.633090 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.632966    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret podName:e693b077-566f-41ce-b702-f9ea0084ed60 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:35.632948519 +0000 UTC m=+4.133668781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret") pod "global-pull-secret-syncer-fqcqr" (UID: "e693b077-566f-41ce-b702-f9ea0084ed60") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:34.733162 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.733123    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4vb\" (UniqueName: \"kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb\") pod \"network-check-target-t8djg\" (UID: \"1a71dff7-c64a-402f-ac3f-a0c2bb078721\") " pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:34.733379 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.733285    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:34.733379 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.733321    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:34.733379 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.733333    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4b4vb for pod openshift-network-diagnostics/network-check-target-t8djg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:34.733533 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:34.733394    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb podName:1a71dff7-c64a-402f-ac3f-a0c2bb078721 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:35.733376133 +0000 UTC m=+4.234096379 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4vb" (UniqueName: "kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb") pod "network-check-target-t8djg" (UID: "1a71dff7-c64a-402f-ac3f-a0c2bb078721") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:34.956841 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.956732    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 11:46:33 +0000 UTC" deadline="2028-05-02 22:04:01.657594359 +0000 UTC"
Sep 11 11:51:34.956841 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:34.956785    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14386h12m26.700812478s"
Sep 11 11:51:35.019220 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:35.019184    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:35.019220 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:35.019203    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:35.019464 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:35.019323    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:35.019464 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:35.019394    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:35.141713 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:35.141531    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1264750e_2181_40ee_8953_fc6c444b9bd5.slice/crio-7a9901576d251924c25ccb07db220763d9a16143ae60f3d1b80851516c67d1cb WatchSource:0}: Error finding container 7a9901576d251924c25ccb07db220763d9a16143ae60f3d1b80851516c67d1cb: Status 404 returned error can't find the container with id 7a9901576d251924c25ccb07db220763d9a16143ae60f3d1b80851516c67d1cb
Sep 11 11:51:35.142918 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:35.142758    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86e098b8_e73f_43c1_bfe3_242bd950b4bd.slice/crio-9c715f77bef64fa7fd2cabb7ccd4b5eb0a86de1c40d58f3f07fe20be8dd35af6 WatchSource:0}: Error finding container 9c715f77bef64fa7fd2cabb7ccd4b5eb0a86de1c40d58f3f07fe20be8dd35af6: Status 404 returned error can't find the container with id 9c715f77bef64fa7fd2cabb7ccd4b5eb0a86de1c40d58f3f07fe20be8dd35af6
Sep 11 11:51:35.145482 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:35.145457    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda24a34f9_b1e8_4418_8f31_3bebcf7dcbfd.slice/crio-87f154c47ce9fa171f92463f0a6af14850d96e2475462b7b5244014bec4ef63e WatchSource:0}: Error finding container 87f154c47ce9fa171f92463f0a6af14850d96e2475462b7b5244014bec4ef63e: Status 404 returned error can't find the container with id 87f154c47ce9fa171f92463f0a6af14850d96e2475462b7b5244014bec4ef63e
Sep 11 11:51:35.148610 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:35.148585    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d25bbd_8989_4fae_912f_329c7904574e.slice/crio-90cdf7cc94f47ba72d303c7721121eb6452927eefd718483344bd853b0ed7df6 WatchSource:0}: Error finding container 90cdf7cc94f47ba72d303c7721121eb6452927eefd718483344bd853b0ed7df6: Status 404 returned error can't find the container with id 90cdf7cc94f47ba72d303c7721121eb6452927eefd718483344bd853b0ed7df6
Sep 11 11:51:35.169715 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:35.169690    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d8a49a_8299_402c_83b2_4de5f94b5552.slice/crio-d411c9792fdb32fb74a767e89eac3d8f7fca99cffe6f79cff7a81d46f064252f WatchSource:0}: Error finding container d411c9792fdb32fb74a767e89eac3d8f7fca99cffe6f79cff7a81d46f064252f: Status 404 returned error can't find the container with id d411c9792fdb32fb74a767e89eac3d8f7fca99cffe6f79cff7a81d46f064252f
Sep 11 11:51:35.171152 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:35.171116    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ce1114_8dd3_4330_a889_4d826243cb38.slice/crio-f012ab21c1e9101c80a1be979cd0eb2d471ff06ceb2868b235b8336ce21fc8f5 WatchSource:0}: Error finding container f012ab21c1e9101c80a1be979cd0eb2d471ff06ceb2868b235b8336ce21fc8f5: Status 404 returned error can't find the container with id f012ab21c1e9101c80a1be979cd0eb2d471ff06ceb2868b235b8336ce21fc8f5
Sep 11 11:51:35.172543 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:35.172516    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c05f73e_008f_497c_9e03_6b411a683a03.slice/crio-8f7b97003896f088c9e50f44440d83ac7e36bb847b0ed53d3ed5f5877e1debea WatchSource:0}: Error finding container 8f7b97003896f088c9e50f44440d83ac7e36bb847b0ed53d3ed5f5877e1debea: Status 404 returned error can't find the container with id 8f7b97003896f088c9e50f44440d83ac7e36bb847b0ed53d3ed5f5877e1debea
Sep 11 11:51:35.173266 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:35.173245    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2970bc44_0b6c_4fc7_89d3_c5db6b9d10c6.slice/crio-69e8f5fbc798a3fb8bd00f139c230e9281c8a63e4925ff05e54aa8ebc7a5d5a6 WatchSource:0}: Error finding container 69e8f5fbc798a3fb8bd00f139c230e9281c8a63e4925ff05e54aa8ebc7a5d5a6: Status 404 returned error can't find the container with id 69e8f5fbc798a3fb8bd00f139c230e9281c8a63e4925ff05e54aa8ebc7a5d5a6
Sep 11 11:51:35.174469 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:51:35.174450    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e028310_ec98_4645_ad5c_48db2103260d.slice/crio-1e13dd14e635c2a711881121796ffa171de5e20c03cf64b715af61e8d44785f3 WatchSource:0}: Error finding container 1e13dd14e635c2a711881121796ffa171de5e20c03cf64b715af61e8d44785f3: Status 404 returned error can't find the container with id 1e13dd14e635c2a711881121796ffa171de5e20c03cf64b715af61e8d44785f3
Sep 11 11:51:35.598598 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:35.598553    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:51:35.639595 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:35.639559    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:35.639750 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:35.639663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:35.639819 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:35.639768    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:35.639870 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:35.639834    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret podName:e693b077-566f-41ce-b702-f9ea0084ed60 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:37.639814197 +0000 UTC m=+6.140534447 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret") pod "global-pull-secret-syncer-fqcqr" (UID: "e693b077-566f-41ce-b702-f9ea0084ed60") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:35.639870 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:35.639770    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:35.639973 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:35.639906    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs podName:142371cb-79c5-44b6-9e7d-9c01359b29f4 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:37.639888932 +0000 UTC m=+6.140609192 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs") pod "network-metrics-daemon-w98wm" (UID: "142371cb-79c5-44b6-9e7d-9c01359b29f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:35.741101 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:35.740457    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4vb\" (UniqueName: \"kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb\") pod \"network-check-target-t8djg\" (UID: \"1a71dff7-c64a-402f-ac3f-a0c2bb078721\") " pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:35.741101 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:35.740629    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:35.741101 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:35.740651    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:35.741101 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:35.740663    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4b4vb for pod openshift-network-diagnostics/network-check-target-t8djg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:35.741101 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:35.740720    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb podName:1a71dff7-c64a-402f-ac3f-a0c2bb078721 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:37.740703052 +0000 UTC m=+6.241423301 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4vb" (UniqueName: "kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb") pod "network-check-target-t8djg" (UID: "1a71dff7-c64a-402f-ac3f-a0c2bb078721") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:35.958514 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:35.957968    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 11:46:33 +0000 UTC" deadline="2028-02-04 18:11:33.54208839 +0000 UTC"
Sep 11 11:51:35.958514 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:35.958009    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12270h19m57.584084056s"
Sep 11 11:51:36.025719 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.025100    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:36.025719 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:36.025248    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:36.042328 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.041770    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal" event={"ID":"0b320866b4a982a72db4f26f5aaaff86","Type":"ContainerStarted","Data":"de2528e42ee2c09d1b4ac978310af03fbcc0d67eda8797a0b3ea2d81579a1d7c"}
Sep 11 11:51:36.046828 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.046790    2562 generic.go:356] "Generic (PLEG): container finished" podID="a890768e232d08590a8e99d453224f11" containerID="eebe1da231d0020eb4a0674cb57dadfa62f6b133c1c7d8ae80c640e64de54acd" exitCode=0
Sep 11 11:51:36.052191 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.052072    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal" event={"ID":"a890768e232d08590a8e99d453224f11","Type":"ContainerDied","Data":"eebe1da231d0020eb4a0674cb57dadfa62f6b133c1c7d8ae80c640e64de54acd"}
Sep 11 11:51:36.055653 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.054453    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp" event={"ID":"f2ce1114-8dd3-4330-a889-4d826243cb38","Type":"ContainerStarted","Data":"f012ab21c1e9101c80a1be979cd0eb2d471ff06ceb2868b235b8336ce21fc8f5"}
Sep 11 11:51:36.058913 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.058830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkpgh" event={"ID":"7c05f73e-008f-497c-9e03-6b411a683a03","Type":"ContainerStarted","Data":"8f7b97003896f088c9e50f44440d83ac7e36bb847b0ed53d3ed5f5877e1debea"}
Sep 11 11:51:36.062519 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.061479    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-222.ec2.internal" podStartSLOduration=3.061463617 podStartE2EDuration="3.061463617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:36.06110444 +0000 UTC m=+4.561824720" watchObservedRunningTime="2026-09-11 11:51:36.061463617 +0000 UTC m=+4.562183886"
Sep 11 11:51:36.066668 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.066642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lv7nr" event={"ID":"1e028310-ec98-4645-ad5c-48db2103260d","Type":"ContainerStarted","Data":"1e13dd14e635c2a711881121796ffa171de5e20c03cf64b715af61e8d44785f3"}
Sep 11 11:51:36.075748 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.075677    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xtfmj" event={"ID":"86e098b8-e73f-43c1-bfe3-242bd950b4bd","Type":"ContainerStarted","Data":"9c715f77bef64fa7fd2cabb7ccd4b5eb0a86de1c40d58f3f07fe20be8dd35af6"}
Sep 11 11:51:36.089189 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.089154    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qnd89" event={"ID":"1264750e-2181-40ee-8953-fc6c444b9bd5","Type":"ContainerStarted","Data":"7a9901576d251924c25ccb07db220763d9a16143ae60f3d1b80851516c67d1cb"}
Sep 11 11:51:36.097020 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.096977    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pxcxn" event={"ID":"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6","Type":"ContainerStarted","Data":"69e8f5fbc798a3fb8bd00f139c230e9281c8a63e4925ff05e54aa8ebc7a5d5a6"}
Sep 11 11:51:36.100419 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.100294    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rfckf" event={"ID":"e0d8a49a-8299-402c-83b2-4de5f94b5552","Type":"ContainerStarted","Data":"d411c9792fdb32fb74a767e89eac3d8f7fca99cffe6f79cff7a81d46f064252f"}
Sep 11 11:51:36.107130 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.107094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg" event={"ID":"e5d25bbd-8989-4fae-912f-329c7904574e","Type":"ContainerStarted","Data":"90cdf7cc94f47ba72d303c7721121eb6452927eefd718483344bd853b0ed7df6"}
Sep 11 11:51:36.110692 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:36.110630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6f2ph" event={"ID":"a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd","Type":"ContainerStarted","Data":"87f154c47ce9fa171f92463f0a6af14850d96e2475462b7b5244014bec4ef63e"}
Sep 11 11:51:37.018779 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:37.018698    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:37.019497 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:37.018808    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:37.019497 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:37.019122    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:37.019497 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:37.019184    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:37.135067 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:37.134996    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal" event={"ID":"a890768e232d08590a8e99d453224f11","Type":"ContainerStarted","Data":"286a93f4d878e3caff7920b5111dfe8368d3c854a0d7b0be617316675a4ad2cb"}
Sep 11 11:51:37.660087 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:37.659384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:37.660087 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:37.659438    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:37.660087 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:37.659589    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:37.660087 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:37.659649    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs podName:142371cb-79c5-44b6-9e7d-9c01359b29f4 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:41.659631041 +0000 UTC m=+10.160351300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs") pod "network-metrics-daemon-w98wm" (UID: "142371cb-79c5-44b6-9e7d-9c01359b29f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:37.660087 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:37.659724    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:37.660087 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:37.659759    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret podName:e693b077-566f-41ce-b702-f9ea0084ed60 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:41.659747261 +0000 UTC m=+10.160467517 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret") pod "global-pull-secret-syncer-fqcqr" (UID: "e693b077-566f-41ce-b702-f9ea0084ed60") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:37.761462 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:37.760799    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4vb\" (UniqueName: \"kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb\") pod \"network-check-target-t8djg\" (UID: \"1a71dff7-c64a-402f-ac3f-a0c2bb078721\") " pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:37.761462 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:37.760953    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:37.761462 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:37.760974    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:37.761462 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:37.760986    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4b4vb for pod openshift-network-diagnostics/network-check-target-t8djg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:37.761462 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:37.761046    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb podName:1a71dff7-c64a-402f-ac3f-a0c2bb078721 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:41.761029385 +0000 UTC m=+10.261749645 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4vb" (UniqueName: "kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb") pod "network-check-target-t8djg" (UID: "1a71dff7-c64a-402f-ac3f-a0c2bb078721") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:38.020953 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:38.020824    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:38.021415 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:38.020979    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:39.019102 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:39.018618    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:39.019102 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:39.018757    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:39.019102 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:39.018898    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:39.019102 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:39.018985    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:40.019154 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:40.019121    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:40.019747 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:40.019265    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:41.018846 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:41.018809    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:41.019037 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:41.018823    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:41.019037 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:41.018964    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:41.019151 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:41.019030    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:41.698946 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:41.698865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:41.698946 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:41.698935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:41.699527 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:41.699010    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:41.699527 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:41.699087    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret podName:e693b077-566f-41ce-b702-f9ea0084ed60 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:49.699066447 +0000 UTC m=+18.199786708 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret") pod "global-pull-secret-syncer-fqcqr" (UID: "e693b077-566f-41ce-b702-f9ea0084ed60") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:41.699527 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:41.699092    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:41.699527 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:41.699157    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs podName:142371cb-79c5-44b6-9e7d-9c01359b29f4 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:49.699138141 +0000 UTC m=+18.199858408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs") pod "network-metrics-daemon-w98wm" (UID: "142371cb-79c5-44b6-9e7d-9c01359b29f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:41.800075 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:41.800026    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4vb\" (UniqueName: \"kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb\") pod \"network-check-target-t8djg\" (UID: \"1a71dff7-c64a-402f-ac3f-a0c2bb078721\") " pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:41.800258 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:41.800209    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:41.800258 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:41.800234    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:41.800258 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:41.800247    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4b4vb for pod openshift-network-diagnostics/network-check-target-t8djg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:41.800421 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:41.800330    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb podName:1a71dff7-c64a-402f-ac3f-a0c2bb078721 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:49.800297111 +0000 UTC m=+18.301017363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4vb" (UniqueName: "kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb") pod "network-check-target-t8djg" (UID: "1a71dff7-c64a-402f-ac3f-a0c2bb078721") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:42.020390 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:42.020174    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:42.020390 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:42.020262    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:43.018455 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:43.018419    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:43.018864 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:43.018554    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:43.018939 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:43.018921    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:43.019048 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:43.019027    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:44.019748 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:44.019537    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:44.019748 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:44.019683    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:45.018995 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:45.018791    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:45.019158 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:45.018862    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:45.019158 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:45.019094    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:45.019362 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:45.019154    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:46.019312 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:46.019203    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:46.019720 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:46.019379    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:47.018364 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:47.018318    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:47.018569 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:47.018436    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:47.018569 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:47.018496    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:47.018690 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:47.018607    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:48.018702 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:48.018665    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:48.019111 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:48.018848    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:49.018982 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:49.018942    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:49.019557 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:49.018943    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:49.019557 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:49.019083    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:49.019557 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:49.019141    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:49.756162 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:49.756074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:49.756162 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:49.756127    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:49.756396 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:49.756230    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:49.756396 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:49.756235    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:49.756396 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:49.756295    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs podName:142371cb-79c5-44b6-9e7d-9c01359b29f4 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:05.756278745 +0000 UTC m=+34.256998994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs") pod "network-metrics-daemon-w98wm" (UID: "142371cb-79c5-44b6-9e7d-9c01359b29f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:51:49.756396 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:49.756326    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret podName:e693b077-566f-41ce-b702-f9ea0084ed60 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:05.756318625 +0000 UTC m=+34.257038878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret") pod "global-pull-secret-syncer-fqcqr" (UID: "e693b077-566f-41ce-b702-f9ea0084ed60") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:51:49.856580 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:49.856542    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4vb\" (UniqueName: \"kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb\") pod \"network-check-target-t8djg\" (UID: \"1a71dff7-c64a-402f-ac3f-a0c2bb078721\") " pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:49.856765 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:49.856742    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:51:49.856847 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:49.856772    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:51:49.856847 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:49.856785    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4b4vb for pod openshift-network-diagnostics/network-check-target-t8djg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:49.856934 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:49.856852    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb podName:1a71dff7-c64a-402f-ac3f-a0c2bb078721 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:05.856832316 +0000 UTC m=+34.357552575 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4vb" (UniqueName: "kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb") pod "network-check-target-t8djg" (UID: "1a71dff7-c64a-402f-ac3f-a0c2bb078721") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:51:50.019324 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:50.019225    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:50.019770 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:50.019397    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:51.018760 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:51.018723    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:51.018933 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:51.018722    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:51.018933 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:51.018879    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:51.019022 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:51.018930    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:52.019317 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:52.019275    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:52.019725 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:52.019393    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:53.018622 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:53.018586    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:53.018768 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:53.018696    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:53.018768 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:53.018752    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:53.018869 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:53.018848    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:54.019423 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.019191    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:54.020292 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:54.019525    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:54.170797 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.170690    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp" event={"ID":"f2ce1114-8dd3-4330-a889-4d826243cb38","Type":"ContainerStarted","Data":"b0e00a6ccef2c73b42208a218bca38ab54a22a88674956e4699ba1f1179a5dab"}
Sep 11 11:51:54.172995 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.172967    2562 generic.go:356] "Generic (PLEG): container finished" podID="7c05f73e-008f-497c-9e03-6b411a683a03" containerID="9c219700122d1661948f2d3f86ee835051664397bcf219d2d9165c6d2672e7b5" exitCode=0
Sep 11 11:51:54.173132 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.173058    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkpgh" event={"ID":"7c05f73e-008f-497c-9e03-6b411a683a03","Type":"ContainerDied","Data":"9c219700122d1661948f2d3f86ee835051664397bcf219d2d9165c6d2672e7b5"}
Sep 11 11:51:54.175226 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.175173    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lv7nr" event={"ID":"1e028310-ec98-4645-ad5c-48db2103260d","Type":"ContainerStarted","Data":"66cb4cb0de77cd6b7b291bcb1698d111c00509b823810d02487448fc4a539417"}
Sep 11 11:51:54.176963 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.176940    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xtfmj" event={"ID":"86e098b8-e73f-43c1-bfe3-242bd950b4bd","Type":"ContainerStarted","Data":"c0e1cfd832020a0920c81d9eef5989c8e72a2a1a2bb6bb7e555f4d107d025fb8"}
Sep 11 11:51:54.178610 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.178583    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pxcxn" event={"ID":"2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6","Type":"ContainerStarted","Data":"30eec853200cc493cd049eed7adddeedf58ca734a8a5aafa0d2f436dd2e5da3b"}
Sep 11 11:51:54.180224 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.180199    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rfckf" event={"ID":"e0d8a49a-8299-402c-83b2-4de5f94b5552","Type":"ContainerStarted","Data":"f591477f1423f4633102a9783561442c564c9506b476fef8e15907c425f6bf30"}
Sep 11 11:51:54.183662 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.183640    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg" event={"ID":"e5d25bbd-8989-4fae-912f-329c7904574e","Type":"ContainerStarted","Data":"47abc4d834a148a1eb930f05110e491c1fd0866ec00c73ef2d2d70d85789abe8"}
Sep 11 11:51:54.183764 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.183717    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg" event={"ID":"e5d25bbd-8989-4fae-912f-329c7904574e","Type":"ContainerStarted","Data":"57cf965db7260d3cc4be6ca0fdae5dcbb054b585c3be491b3088a4bc3d82119c"}
Sep 11 11:51:54.183825 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.183770    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg" event={"ID":"e5d25bbd-8989-4fae-912f-329c7904574e","Type":"ContainerStarted","Data":"f6d599aca2f3ed79c600892b59c7d1781415c4bc6b544d6ef175936856bb50a7"}
Sep 11 11:51:54.183825 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.183820    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg" event={"ID":"e5d25bbd-8989-4fae-912f-329c7904574e","Type":"ContainerStarted","Data":"5ef84856b765ded5701efddffcb9ea3122a7e5c796a6effcc486d6f735fac9c7"}
Sep 11 11:51:54.183925 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.183870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg" event={"ID":"e5d25bbd-8989-4fae-912f-329c7904574e","Type":"ContainerStarted","Data":"23dce4dfe3f3e0035f6fbd8d59d37c881382c03fe04fc085e5c48c58e33fe272"}
Sep 11 11:51:54.183925 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.183920    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg" event={"ID":"e5d25bbd-8989-4fae-912f-329c7904574e","Type":"ContainerStarted","Data":"b1c887bbeb19e78e96451c4fc1700f6bae1ef79c620c90c90f2b09f24e7bd1a4"}
Sep 11 11:51:54.185318 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.185244    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6f2ph" event={"ID":"a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd","Type":"ContainerStarted","Data":"73717878239eabcf26472abb9ad6ffdaa7e0def573401f78856d4afc0aef1f8b"}
Sep 11 11:51:54.198350 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.198292    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-222.ec2.internal" podStartSLOduration=21.198280199 podStartE2EDuration="21.198280199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:37.150182374 +0000 UTC m=+5.650902643" watchObservedRunningTime="2026-09-11 11:51:54.198280199 +0000 UTC m=+22.699000468"
Sep 11 11:51:54.216156 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.216104    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6f2ph" podStartSLOduration=8.04885384 podStartE2EDuration="22.216085377s" totalImagesPullingTime="14.167231537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:54.215602423 +0000 UTC m=+22.716322692" watchObservedRunningTime="2026-09-11 11:51:54.216085377 +0000 UTC m=+22.716805644"
Sep 11 11:51:54.216348 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.216316    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lv7nr" podStartSLOduration=4.220037744 podStartE2EDuration="22.216295131s" totalImagesPullingTime="17.996257387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:54.20329944 +0000 UTC m=+22.704019707" watchObservedRunningTime="2026-09-11 11:51:54.216295131 +0000 UTC m=+22.717015398"
Sep 11 11:51:54.229708 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.229661    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pxcxn" podStartSLOduration=4.180651256 podStartE2EDuration="22.229649603s" totalImagesPullingTime="18.048998347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:54.229188445 +0000 UTC m=+22.729908713" watchObservedRunningTime="2026-09-11 11:51:54.229649603 +0000 UTC m=+22.730369870"
Sep 11 11:51:54.243752 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.243704    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rfckf" podStartSLOduration=4.232785447 podStartE2EDuration="22.243690253s" totalImagesPullingTime="18.010904806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:54.243046357 +0000 UTC m=+22.743766624" watchObservedRunningTime="2026-09-11 11:51:54.243690253 +0000 UTC m=+22.744410520"
Sep 11 11:51:54.260068 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.260027    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xtfmj" podStartSLOduration=4.230586559 podStartE2EDuration="22.26001216s" totalImagesPullingTime="18.029425601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:54.259939244 +0000 UTC m=+22.760659513" watchObservedRunningTime="2026-09-11 11:51:54.26001216 +0000 UTC m=+22.760732428"
Sep 11 11:51:54.654063 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.653904    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 11:51:54.993410 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.993249    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T11:51:54.654057794Z","UUID":"8fe2057c-55bd-496e-8939-703532ba475b","Handler":null,"Name":"","Endpoint":""}
Sep 11 11:51:54.994817 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.994768    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 11:51:54.994817 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:54.994799    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 11:51:55.018533 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:55.018500    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:55.018769 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:55.018545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:55.018967 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:55.018933    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:55.019139 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:55.019054    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:55.189569 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:55.189535    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp" event={"ID":"f2ce1114-8dd3-4330-a889-4d826243cb38","Type":"ContainerStarted","Data":"cc4be0d4426efce5e83154018d41a8effe91d77c7706210f09257aeb4ce9d504"}
Sep 11 11:51:55.191586 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:55.191431    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qnd89" event={"ID":"1264750e-2181-40ee-8953-fc6c444b9bd5","Type":"ContainerStarted","Data":"18a1daf8f5efe64c33518891c3e2a3c8320c65bad80bcc5ae93ee1b96c322517"}
Sep 11 11:51:55.204266 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:55.204202    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qnd89" podStartSLOduration=5.17828866 podStartE2EDuration="23.204179988s" totalImagesPullingTime="18.025891328s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:55.2033927 +0000 UTC m=+23.704112970" watchObservedRunningTime="2026-09-11 11:51:55.204179988 +0000 UTC m=+23.704900257"
Sep 11 11:51:56.019549 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:56.019085    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:56.019549 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:56.019239    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:56.198067 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:56.198033    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg" event={"ID":"e5d25bbd-8989-4fae-912f-329c7904574e","Type":"ContainerStarted","Data":"d5c27627b5f85cb297231b422309ef6912ed5a0d6f0caac20e0a7b4f587329e5"}
Sep 11 11:51:57.018982 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:57.018951    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:57.019187 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:57.018957    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:57.019187 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:57.019062    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:57.019187 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:57.019115    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:57.203326 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:57.203275    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp" event={"ID":"f2ce1114-8dd3-4330-a889-4d826243cb38","Type":"ContainerStarted","Data":"1fc90b634f8ed07cb90ae56f079f53563a6c7cf3ed86a85e5265a592b6c86b61"}
Sep 11 11:51:57.219461 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:57.219397    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rqjmp" podStartSLOduration=4.661375164 podStartE2EDuration="25.219379139s" totalImagesPullingTime="20.558003975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:57.218146733 +0000 UTC m=+25.718867003" watchObservedRunningTime="2026-09-11 11:51:57.219379139 +0000 UTC m=+25.720099556"
Sep 11 11:51:58.018983 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:58.018946    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:51:58.019140 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:58.019102    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:51:58.365851 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:58.365762    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:58.366563 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:58.366546    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:59.019068 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:59.018821    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:51:59.019235 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:59.018848    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:51:59.019235 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:59.019190    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:51:59.019333 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:51:59.019235    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:51:59.210696 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:59.210667    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkpgh" event={"ID":"7c05f73e-008f-497c-9e03-6b411a683a03","Type":"ContainerStarted","Data":"3a6456d66f8d8a2fb14bce01075315930d909e99d280d0c4b4728c563c0352c4"}
Sep 11 11:51:59.214436 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:59.214409    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg" event={"ID":"e5d25bbd-8989-4fae-912f-329c7904574e","Type":"ContainerStarted","Data":"fbdc1c433398a2aea298e1f2bcdc3a4a605270f6065961879240f2aefccea971"}
Sep 11 11:51:59.214763 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:59.214742    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:59.215167 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:59.215146    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6f2ph"
Sep 11 11:51:59.236669 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:51:59.236626    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg" podStartSLOduration=9.177869666 podStartE2EDuration="27.236613444s" totalImagesPullingTime="18.058743778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:59.235745026 +0000 UTC m=+27.736465294" watchObservedRunningTime="2026-09-11 11:51:59.236613444 +0000 UTC m=+27.737333711"
Sep 11 11:52:00.019234 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.019195    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:52:00.020032 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:00.019324    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:52:00.218274 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.218238    2562 generic.go:356] "Generic (PLEG): container finished" podID="7c05f73e-008f-497c-9e03-6b411a683a03" containerID="3a6456d66f8d8a2fb14bce01075315930d909e99d280d0c4b4728c563c0352c4" exitCode=0
Sep 11 11:52:00.218441 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.218343    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkpgh" event={"ID":"7c05f73e-008f-497c-9e03-6b411a683a03","Type":"ContainerDied","Data":"3a6456d66f8d8a2fb14bce01075315930d909e99d280d0c4b4728c563c0352c4"}
Sep 11 11:52:00.219526 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.219120    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:52:00.219526 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.219206    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:52:00.219526 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.219256    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:52:00.233864 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.233834    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:52:00.236598 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.236560    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:52:00.737475 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.737235    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w98wm"]
Sep 11 11:52:00.737625 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.737547    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:52:00.737680 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:00.737658    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:52:00.739046 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.739008    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t8djg"]
Sep 11 11:52:00.739182 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.739119    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:52:00.739257 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:00.739222    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:52:00.739800 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.739768    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fqcqr"]
Sep 11 11:52:00.739908 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:00.739860    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:52:00.739972 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:00.739944    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:52:01.567958 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:01.567925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:01.645267 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:01.645232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:02.019739 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:02.019701    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:52:02.019924 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:02.019817    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:52:02.019924 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:02.019835    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:52:02.020061 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:02.019932    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:52:03.019372 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:03.019337    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:52:03.019851 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:03.019468    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:52:03.154320 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:03.154227    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:03.215856 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:03.215822    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:03.227408 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:03.227371    2562 generic.go:356] "Generic (PLEG): container finished" podID="7c05f73e-008f-497c-9e03-6b411a683a03" containerID="11f8bc1833e9f08c78389a2c285254a6378d8949e1dd6ec7148324237946eb66" exitCode=0
Sep 11 11:52:03.227515 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:03.227430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkpgh" event={"ID":"7c05f73e-008f-497c-9e03-6b411a683a03","Type":"ContainerDied","Data":"11f8bc1833e9f08c78389a2c285254a6378d8949e1dd6ec7148324237946eb66"}
Sep 11 11:52:03.488080 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:03.488046    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:03.539437 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:03.539385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:04.018873 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:04.018843    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:52:04.018995 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:04.018947    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8djg" podUID="1a71dff7-c64a-402f-ac3f-a0c2bb078721"
Sep 11 11:52:04.019071 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:04.019003    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:52:04.019147 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:04.019124    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fqcqr" podUID="e693b077-566f-41ce-b702-f9ea0084ed60"
Sep 11 11:52:04.231949 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:04.231861    2562 generic.go:356] "Generic (PLEG): container finished" podID="7c05f73e-008f-497c-9e03-6b411a683a03" containerID="15915811293dce18e64aeb1faeb5ac8225f952de229669ba7c98a76b56bd7e9b" exitCode=0
Sep 11 11:52:04.231949 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:04.231929    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkpgh" event={"ID":"7c05f73e-008f-497c-9e03-6b411a683a03","Type":"ContainerDied","Data":"15915811293dce18e64aeb1faeb5ac8225f952de229669ba7c98a76b56bd7e9b"}
Sep 11 11:52:05.018602 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.018560    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:52:05.018779 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:05.018706    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w98wm" podUID="142371cb-79c5-44b6-9e7d-9c01359b29f4"
Sep 11 11:52:05.721845 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.721799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:05.772517 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.772485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:05.776545 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.776521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:52:05.776692 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.776563    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:52:05.776753 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:05.776684    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:52:05.776753 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:05.776712    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:52:05.776753 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:05.776751    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret podName:e693b077-566f-41ce-b702-f9ea0084ed60 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:37.776730385 +0000 UTC m=+66.277450646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret") pod "global-pull-secret-syncer-fqcqr" (UID: "e693b077-566f-41ce-b702-f9ea0084ed60") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:52:05.776881 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:05.776773    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs podName:142371cb-79c5-44b6-9e7d-9c01359b29f4 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:37.776762162 +0000 UTC m=+66.277482422 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs") pod "network-metrics-daemon-w98wm" (UID: "142371cb-79c5-44b6-9e7d-9c01359b29f4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:52:05.857028 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.857003    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-222.ec2.internal" event="NodeReady"
Sep 11 11:52:05.857186 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.857153    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 11:52:05.876944 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.876915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4vb\" (UniqueName: \"kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb\") pod \"network-check-target-t8djg\" (UID: \"1a71dff7-c64a-402f-ac3f-a0c2bb078721\") " pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:52:05.877092 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:05.877056    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:52:05.877092 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:05.877072    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:52:05.877092 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:05.877086    2562 projected.go:196] Error preparing data for projected volume kube-api-access-4b4vb for pod openshift-network-diagnostics/network-check-target-t8djg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:52:05.877227 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:05.877146    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb podName:1a71dff7-c64a-402f-ac3f-a0c2bb078721 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:37.877128304 +0000 UTC m=+66.377848569 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4vb" (UniqueName: "kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb") pod "network-check-target-t8djg" (UID: "1a71dff7-c64a-402f-ac3f-a0c2bb078721") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:52:05.898716 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.898686    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jsxzt"]
Sep 11 11:52:05.901392 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.901369    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-d5cxg"]
Sep 11 11:52:05.901527 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.901490    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:05.903367 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.903342    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 11:52:05.903367 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.903366    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 11:52:05.903527 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.903449    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mvnq6\""
Sep 11 11:52:05.904333 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.904298    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d5cxg"
Sep 11 11:52:05.906101 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.906084    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 11:52:05.906101 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.906092    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 11:52:05.906238 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.906122    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 11:52:05.906238 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.906124    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5ncdh\""
Sep 11 11:52:05.908564 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.908543    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d5cxg"]
Sep 11 11:52:05.911522 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.911500    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jsxzt"]
Sep 11 11:52:05.977482 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.977401    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2vcm\" (UniqueName: \"kubernetes.io/projected/1d546130-4201-41ee-ba52-d2f1d1b9e205-kube-api-access-h2vcm\") pod \"ingress-canary-d5cxg\" (UID: \"1d546130-4201-41ee-ba52-d2f1d1b9e205\") " pod="openshift-ingress-canary/ingress-canary-d5cxg"
Sep 11 11:52:05.977482 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.977478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/28c2284e-89be-4c9f-a804-e1a2373a8f92-config-volume\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:05.977687 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.977509    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28c2284e-89be-4c9f-a804-e1a2373a8f92-tmp-dir\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:05.977687 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.977545    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-829wg\" (UniqueName: \"kubernetes.io/projected/28c2284e-89be-4c9f-a804-e1a2373a8f92-kube-api-access-829wg\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:05.977687 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.977632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:05.977687 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:05.977676    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d546130-4201-41ee-ba52-d2f1d1b9e205-cert\") pod \"ingress-canary-d5cxg\" (UID: \"1d546130-4201-41ee-ba52-d2f1d1b9e205\") " pod="openshift-ingress-canary/ingress-canary-d5cxg"
Sep 11 11:52:06.014569 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.014537    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:06.022191 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.022169    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:52:06.022191 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.022191    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:52:06.024245 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.024225    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 11:52:06.024380 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.024256    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 11:52:06.024380 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.024283    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f8bwk\""
Sep 11 11:52:06.024380 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.024298    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 11:52:06.079003 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.078968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2vcm\" (UniqueName: \"kubernetes.io/projected/1d546130-4201-41ee-ba52-d2f1d1b9e205-kube-api-access-h2vcm\") pod \"ingress-canary-d5cxg\" (UID: \"1d546130-4201-41ee-ba52-d2f1d1b9e205\") " pod="openshift-ingress-canary/ingress-canary-d5cxg"
Sep 11 11:52:06.079173 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.079040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/28c2284e-89be-4c9f-a804-e1a2373a8f92-config-volume\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:06.079173 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.079071    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28c2284e-89be-4c9f-a804-e1a2373a8f92-tmp-dir\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:06.079173 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.079104    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-829wg\" (UniqueName: \"kubernetes.io/projected/28c2284e-89be-4c9f-a804-e1a2373a8f92-kube-api-access-829wg\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:06.079173 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.079144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:06.079173 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.079170    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d546130-4201-41ee-ba52-d2f1d1b9e205-cert\") pod \"ingress-canary-d5cxg\" (UID: \"1d546130-4201-41ee-ba52-d2f1d1b9e205\") " pod="openshift-ingress-canary/ingress-canary-d5cxg"
Sep 11 11:52:06.079425 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:06.079349    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:06.079425 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:06.079417    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls podName:28c2284e-89be-4c9f-a804-e1a2373a8f92 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:06.5793987 +0000 UTC m=+35.080118961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls") pod "dns-default-jsxzt" (UID: "28c2284e-89be-4c9f-a804-e1a2373a8f92") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:06.079730 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.079705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28c2284e-89be-4c9f-a804-e1a2373a8f92-tmp-dir\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:06.079866 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.079847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/28c2284e-89be-4c9f-a804-e1a2373a8f92-config-volume\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:06.084209 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.084049    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d546130-4201-41ee-ba52-d2f1d1b9e205-cert\") pod \"ingress-canary-d5cxg\" (UID: \"1d546130-4201-41ee-ba52-d2f1d1b9e205\") " pod="openshift-ingress-canary/ingress-canary-d5cxg"
Sep 11 11:52:06.089028 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.089000    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-829wg\" (UniqueName: \"kubernetes.io/projected/28c2284e-89be-4c9f-a804-e1a2373a8f92-kube-api-access-829wg\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:06.089576 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.089552    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2vcm\" (UniqueName: \"kubernetes.io/projected/1d546130-4201-41ee-ba52-d2f1d1b9e205-kube-api-access-h2vcm\") pod \"ingress-canary-d5cxg\" (UID: \"1d546130-4201-41ee-ba52-d2f1d1b9e205\") " pod="openshift-ingress-canary/ingress-canary-d5cxg"
Sep 11 11:52:06.144800 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.144748    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:06.218995 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.218959    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d5cxg"
Sep 11 11:52:06.511342 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.511226    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d5cxg"]
Sep 11 11:52:06.514972 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:52:06.514940    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d546130_4201_41ee_ba52_d2f1d1b9e205.slice/crio-6765b722143fd11e3cac9020cf1c363e5ca55c4a8e0f0cff4ead5882b8f582f2 WatchSource:0}: Error finding container 6765b722143fd11e3cac9020cf1c363e5ca55c4a8e0f0cff4ead5882b8f582f2: Status 404 returned error can't find the container with id 6765b722143fd11e3cac9020cf1c363e5ca55c4a8e0f0cff4ead5882b8f582f2
Sep 11 11:52:06.582261 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:06.582229    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:06.582463 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:06.582411    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:06.582519 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:06.582482    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls podName:28c2284e-89be-4c9f-a804-e1a2373a8f92 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:07.582462571 +0000 UTC m=+36.083182838 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls") pod "dns-default-jsxzt" (UID: "28c2284e-89be-4c9f-a804-e1a2373a8f92") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:07.018498 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:07.018466    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:52:07.020788 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:07.020765    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 11:52:07.021319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:07.021281    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-74fqm\""
Sep 11 11:52:07.240699 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:07.240663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d5cxg" event={"ID":"1d546130-4201-41ee-ba52-d2f1d1b9e205","Type":"ContainerStarted","Data":"6765b722143fd11e3cac9020cf1c363e5ca55c4a8e0f0cff4ead5882b8f582f2"}
Sep 11 11:52:07.589568 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:07.589518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:07.589737 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:07.589686    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:07.589793 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:07.589772    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls podName:28c2284e-89be-4c9f-a804-e1a2373a8f92 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:09.589750745 +0000 UTC m=+38.090470993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls") pod "dns-default-jsxzt" (UID: "28c2284e-89be-4c9f-a804-e1a2373a8f92") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:08.160666 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:08.160629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:08.261202 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:08.261163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:09.249281 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:09.249244    2562 generic.go:356] "Generic (PLEG): container finished" podID="7c05f73e-008f-497c-9e03-6b411a683a03" containerID="593fed8e112228257de3d38b32dcd63e1754f1d72ae02301c27c17d12547637e" exitCode=0
Sep 11 11:52:09.249810 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:09.249322    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkpgh" event={"ID":"7c05f73e-008f-497c-9e03-6b411a683a03","Type":"ContainerDied","Data":"593fed8e112228257de3d38b32dcd63e1754f1d72ae02301c27c17d12547637e"}
Sep 11 11:52:09.606787 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:09.606494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:09.606787 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:09.606675    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:09.607004 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:09.606803    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls podName:28c2284e-89be-4c9f-a804-e1a2373a8f92 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:13.606780115 +0000 UTC m=+42.107500362 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls") pod "dns-default-jsxzt" (UID: "28c2284e-89be-4c9f-a804-e1a2373a8f92") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:10.254767 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:10.254735    2562 generic.go:356] "Generic (PLEG): container finished" podID="7c05f73e-008f-497c-9e03-6b411a683a03" containerID="5e54c09ab64e9f206198c6e709f754feded439db2154f7b2e9a57a2037617a42" exitCode=0
Sep 11 11:52:10.255133 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:10.254785    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkpgh" event={"ID":"7c05f73e-008f-497c-9e03-6b411a683a03","Type":"ContainerDied","Data":"5e54c09ab64e9f206198c6e709f754feded439db2154f7b2e9a57a2037617a42"}
Sep 11 11:52:11.258925 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:11.258881    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d5cxg" event={"ID":"1d546130-4201-41ee-ba52-d2f1d1b9e205","Type":"ContainerStarted","Data":"450a99c0d0f10b65d88e257977b1462fab4a7cf8e65a04890b9ba4e7062cfdb8"}
Sep 11 11:52:11.262531 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:11.262501    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkpgh" event={"ID":"7c05f73e-008f-497c-9e03-6b411a683a03","Type":"ContainerStarted","Data":"1fa9c37e41fafaef670e4487ceeb1b703c89fcd2ea27a9fb15c5614068e6337c"}
Sep 11 11:52:11.273070 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:11.273025    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-d5cxg" podStartSLOduration=2.312374748 podStartE2EDuration="6.273011764s" totalImagesPullingTime="3.960637016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:11.271995042 +0000 UTC m=+39.772715309" watchObservedRunningTime="2026-09-11 11:52:11.273011764 +0000 UTC m=+39.773732031"
Sep 11 11:52:11.290531 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:11.290481    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nkpgh" podStartSLOduration=8.998514218 podStartE2EDuration="39.290468479s" totalImagesPullingTime="30.291954261s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:11.289654949 +0000 UTC m=+39.790375219" watchObservedRunningTime="2026-09-11 11:52:11.290468479 +0000 UTC m=+39.791188746"
Sep 11 11:52:13.635179 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:13.635144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:13.635595 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:13.635273    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:13.635595 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:13.635349    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls podName:28c2284e-89be-4c9f-a804-e1a2373a8f92 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:21.635336286 +0000 UTC m=+50.136056532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls") pod "dns-default-jsxzt" (UID: "28c2284e-89be-4c9f-a804-e1a2373a8f92") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:21.690287 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:21.690238    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:21.690742 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:21.690405    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:21.690742 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:21.690486    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls podName:28c2284e-89be-4c9f-a804-e1a2373a8f92 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:37.690469738 +0000 UTC m=+66.191189988 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls") pod "dns-default-jsxzt" (UID: "28c2284e-89be-4c9f-a804-e1a2373a8f92") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:28.839273 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.839238    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"]
Sep 11 11:52:28.841356 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.841340    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:28.843425 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.843395    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 11:52:28.843425 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.843405    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 11:52:28.843581 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.843399    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:52:28.843581 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.843399    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 11:52:28.843792 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.843777    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jbmp7\""
Sep 11 11:52:28.852381 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.852362    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"]
Sep 11 11:52:28.940244 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.940209    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-znzgz"]
Sep 11 11:52:28.941646 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.941622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q68qm\" (UniqueName: \"kubernetes.io/projected/a9479947-3863-4fcd-8b58-ded0146677a9-kube-api-access-q68qm\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wqtf9\" (UID: \"a9479947-3863-4fcd-8b58-ded0146677a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:28.941763 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.941656    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9479947-3863-4fcd-8b58-ded0146677a9-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wqtf9\" (UID: \"a9479947-3863-4fcd-8b58-ded0146677a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:28.941763 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.941704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9479947-3863-4fcd-8b58-ded0146677a9-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wqtf9\" (UID: \"a9479947-3863-4fcd-8b58-ded0146677a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:28.942429 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.942409    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:28.944711 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.944688    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 11:52:28.944812 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.944766    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 11:52:28.944812 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.944770    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 11:52:28.944925 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.944888    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sm6rl\""
Sep 11 11:52:28.944980 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.944943    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 11:52:28.949626 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.949606    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 11:52:28.956206 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:28.956179    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-znzgz"]
Sep 11 11:52:29.042536 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.042503    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9479947-3863-4fcd-8b58-ded0146677a9-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wqtf9\" (UID: \"a9479947-3863-4fcd-8b58-ded0146677a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:29.042696 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.042554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09200696-2c09-4c98-8b6e-6d3116c2319a-tmp\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.042696 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.042581    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09200696-2c09-4c98-8b6e-6d3116c2319a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.042696 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.042652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms78q\" (UniqueName: \"kubernetes.io/projected/09200696-2c09-4c98-8b6e-6d3116c2319a-kube-api-access-ms78q\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.042696 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.042688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09200696-2c09-4c98-8b6e-6d3116c2319a-service-ca-bundle\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.042827 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.042719    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09200696-2c09-4c98-8b6e-6d3116c2319a-serving-cert\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.042827 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.042781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/09200696-2c09-4c98-8b6e-6d3116c2319a-snapshots\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.042827 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.042813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q68qm\" (UniqueName: \"kubernetes.io/projected/a9479947-3863-4fcd-8b58-ded0146677a9-kube-api-access-q68qm\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wqtf9\" (UID: \"a9479947-3863-4fcd-8b58-ded0146677a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:29.042917 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.042846    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9479947-3863-4fcd-8b58-ded0146677a9-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wqtf9\" (UID: \"a9479947-3863-4fcd-8b58-ded0146677a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:29.043952 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.043927    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9479947-3863-4fcd-8b58-ded0146677a9-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wqtf9\" (UID: \"a9479947-3863-4fcd-8b58-ded0146677a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:29.045897 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.045880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9479947-3863-4fcd-8b58-ded0146677a9-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wqtf9\" (UID: \"a9479947-3863-4fcd-8b58-ded0146677a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:29.050782 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.050754    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"]
Sep 11 11:52:29.052852 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.052836    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-dfb4b6887-t9bg7"]
Sep 11 11:52:29.052969 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.052951    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:29.054736 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.054718    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 11:52:29.054830 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.054756    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 11:52:29.054885 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.054863    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.054941 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.054881    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bp9zp\""
Sep 11 11:52:29.056264 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.056240    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q68qm\" (UniqueName: \"kubernetes.io/projected/a9479947-3863-4fcd-8b58-ded0146677a9-kube-api-access-q68qm\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wqtf9\" (UID: \"a9479947-3863-4fcd-8b58-ded0146677a9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:29.056699 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.056684    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6j76c\""
Sep 11 11:52:29.056878 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.056866    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 11:52:29.057192 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.057175    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 11:52:29.057250 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.057234    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 11:52:29.062459 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.062440    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 11:52:29.063490 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.063463    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"]
Sep 11 11:52:29.067679 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.067659    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dfb4b6887-t9bg7"]
Sep 11 11:52:29.143870 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.143786    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09200696-2c09-4c98-8b6e-6d3116c2319a-service-ca-bundle\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.143870 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.143821    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09200696-2c09-4c98-8b6e-6d3116c2319a-serving-cert\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.144075 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.143884    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/09200696-2c09-4c98-8b6e-6d3116c2319a-snapshots\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.144075 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.143913    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:29.144075 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.143988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09200696-2c09-4c98-8b6e-6d3116c2319a-tmp\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.144075 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.144009    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09200696-2c09-4c98-8b6e-6d3116c2319a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.144075 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.144031    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:29.144299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.144100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms78q\" (UniqueName: \"kubernetes.io/projected/09200696-2c09-4c98-8b6e-6d3116c2319a-kube-api-access-ms78q\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.144528 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.144504    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09200696-2c09-4c98-8b6e-6d3116c2319a-tmp\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.144603 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.144542    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09200696-2c09-4c98-8b6e-6d3116c2319a-service-ca-bundle\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.144639 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.144618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/09200696-2c09-4c98-8b6e-6d3116c2319a-snapshots\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.144826 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.144809    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09200696-2c09-4c98-8b6e-6d3116c2319a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.146359 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.146342    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09200696-2c09-4c98-8b6e-6d3116c2319a-serving-cert\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.149538 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.149504    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"
Sep 11 11:52:29.151649 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.151631    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms78q\" (UniqueName: \"kubernetes.io/projected/09200696-2c09-4c98-8b6e-6d3116c2319a-kube-api-access-ms78q\") pod \"insights-operator-5844b856db-znzgz\" (UID: \"09200696-2c09-4c98-8b6e-6d3116c2319a\") " pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.245351 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.245297    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-installation-pull-secrets\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.245526 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.245366    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:29.245526 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.245387    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cl98\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-kube-api-access-5cl98\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.245526 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.245410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-trusted-ca\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.245526 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.245483    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-image-registry-private-configuration\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.245660 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.245530    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-ca-trust-extracted\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.245660 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.245563    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:29.245660 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.245589    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.245660 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.245608    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-certificates\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.245660 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.245624    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-bound-sa-token\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.245660 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:29.245642    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:52:29.245874 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:29.245712    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert podName:b669f014-1829-47a8-9d3a-bbb01d4d6eb6 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:29.745697849 +0000 UTC m=+58.246418096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-zq4mw" (UID: "b669f014-1829-47a8-9d3a-bbb01d4d6eb6") : secret "networking-console-plugin-cert" not found
Sep 11 11:52:29.246025 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.246009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:29.250849 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.250834    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-znzgz"
Sep 11 11:52:29.346404 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.346366    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-ca-trust-extracted\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.346574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.346419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.346574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.346440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-certificates\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.346574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.346458    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-bound-sa-token\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.346574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.346486    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-installation-pull-secrets\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.346574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.346530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cl98\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-kube-api-access-5cl98\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.346574 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.346551    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-trusted-ca\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.346574 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:29.346550    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 11:52:29.346910 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:29.346587    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfb4b6887-t9bg7: secret "image-registry-tls" not found
Sep 11 11:52:29.346910 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:29.346690    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls podName:e4a02f19-7ae9-4d6c-850e-da97d5e406b0 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:29.846670834 +0000 UTC m=+58.347391093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls") pod "image-registry-dfb4b6887-t9bg7" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0") : secret "image-registry-tls" not found
Sep 11 11:52:29.346910 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.346738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-image-registry-private-configuration\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.346910 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.346846    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-ca-trust-extracted\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.347172 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.347139    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-certificates\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.347546 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.347529    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-trusted-ca\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.348821 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.348804    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-installation-pull-secrets\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.348992 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.348977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-image-registry-private-configuration\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.356123 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.356098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-bound-sa-token\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.356257 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.356242    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cl98\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-kube-api-access-5cl98\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.419607 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.419532    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9"]
Sep 11 11:52:29.422475 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:52:29.422448    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9479947_3863_4fcd_8b58_ded0146677a9.slice/crio-9bede0a22629d89c525aaeb663172bd3cfc78de1b00a2df933319a9171b92beb WatchSource:0}: Error finding container 9bede0a22629d89c525aaeb663172bd3cfc78de1b00a2df933319a9171b92beb: Status 404 returned error can't find the container with id 9bede0a22629d89c525aaeb663172bd3cfc78de1b00a2df933319a9171b92beb
Sep 11 11:52:29.516492 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.516455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-znzgz"]
Sep 11 11:52:29.520408 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:52:29.520383    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09200696_2c09_4c98_8b6e_6d3116c2319a.slice/crio-67f2726e3148b829e5895f3eeeaba4f0b8405adfa27bcf997d7bb3a9c9d97868 WatchSource:0}: Error finding container 67f2726e3148b829e5895f3eeeaba4f0b8405adfa27bcf997d7bb3a9c9d97868: Status 404 returned error can't find the container with id 67f2726e3148b829e5895f3eeeaba4f0b8405adfa27bcf997d7bb3a9c9d97868
Sep 11 11:52:29.749998 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.749960    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:29.750174 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:29.750116    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:52:29.750222 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:29.750184    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert podName:b669f014-1829-47a8-9d3a-bbb01d4d6eb6 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:30.750169352 +0000 UTC m=+59.250889627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-zq4mw" (UID: "b669f014-1829-47a8-9d3a-bbb01d4d6eb6") : secret "networking-console-plugin-cert" not found
Sep 11 11:52:29.850644 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:29.850607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:29.851007 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:29.850725    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 11:52:29.851007 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:29.850736    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfb4b6887-t9bg7: secret "image-registry-tls" not found
Sep 11 11:52:29.851007 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:29.850811    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls podName:e4a02f19-7ae9-4d6c-850e-da97d5e406b0 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:30.850796422 +0000 UTC m=+59.351516682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls") pod "image-registry-dfb4b6887-t9bg7" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0") : secret "image-registry-tls" not found
Sep 11 11:52:30.309929 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:30.309884    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-znzgz" event={"ID":"09200696-2c09-4c98-8b6e-6d3116c2319a","Type":"ContainerStarted","Data":"67f2726e3148b829e5895f3eeeaba4f0b8405adfa27bcf997d7bb3a9c9d97868"}
Sep 11 11:52:30.311299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:30.311253    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9" event={"ID":"a9479947-3863-4fcd-8b58-ded0146677a9","Type":"ContainerStarted","Data":"9bede0a22629d89c525aaeb663172bd3cfc78de1b00a2df933319a9171b92beb"}
Sep 11 11:52:30.757518 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:30.757481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:30.757746 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:30.757636    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:52:30.757746 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:30.757708    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert podName:b669f014-1829-47a8-9d3a-bbb01d4d6eb6 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:32.757692205 +0000 UTC m=+61.258412455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-zq4mw" (UID: "b669f014-1829-47a8-9d3a-bbb01d4d6eb6") : secret "networking-console-plugin-cert" not found
Sep 11 11:52:30.858443 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:30.858405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:30.858894 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:30.858566    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 11:52:30.858894 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:30.858589    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfb4b6887-t9bg7: secret "image-registry-tls" not found
Sep 11 11:52:30.858894 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:30.858655    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls podName:e4a02f19-7ae9-4d6c-850e-da97d5e406b0 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:32.858638788 +0000 UTC m=+61.359359038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls") pod "image-registry-dfb4b6887-t9bg7" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0") : secret "image-registry-tls" not found
Sep 11 11:52:32.234059 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:32.234026    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f8cvg"
Sep 11 11:52:32.317163 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:32.317127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-znzgz" event={"ID":"09200696-2c09-4c98-8b6e-6d3116c2319a","Type":"ContainerStarted","Data":"bb2608d1a1821a7780ab61013f11309c1f957ef660e162cc546448b92ad3c9c8"}
Sep 11 11:52:32.318746 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:32.318722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9" event={"ID":"a9479947-3863-4fcd-8b58-ded0146677a9","Type":"ContainerStarted","Data":"96216c9f29256ca8d02629125a9d5aeeb3b2b2a82b64f42978d72ca992771dc3"}
Sep 11 11:52:32.333657 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:32.333616    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-znzgz" podStartSLOduration=1.975019753 podStartE2EDuration="4.333602039s" totalImagesPullingTime="2.358582286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:32.333350874 +0000 UTC m=+60.834071141" watchObservedRunningTime="2026-09-11 11:52:32.333602039 +0000 UTC m=+60.834322318"
Sep 11 11:52:32.351053 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:32.350998    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9" podStartSLOduration=1.891679514 podStartE2EDuration="4.35095056s" totalImagesPullingTime="2.459271046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:32.350169442 +0000 UTC m=+60.850889710" watchObservedRunningTime="2026-09-11 11:52:32.35095056 +0000 UTC m=+60.851670829"
Sep 11 11:52:32.773943 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:32.773910    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:32.774107 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:32.774067    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:52:32.774169 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:32.774133    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert podName:b669f014-1829-47a8-9d3a-bbb01d4d6eb6 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:36.774117586 +0000 UTC m=+65.274837845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-zq4mw" (UID: "b669f014-1829-47a8-9d3a-bbb01d4d6eb6") : secret "networking-console-plugin-cert" not found
Sep 11 11:52:32.875028 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:32.874991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:32.875197 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:32.875167    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 11:52:32.875197 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:32.875191    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfb4b6887-t9bg7: secret "image-registry-tls" not found
Sep 11 11:52:32.875272 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:32.875260    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls podName:e4a02f19-7ae9-4d6c-850e-da97d5e406b0 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:36.875238742 +0000 UTC m=+65.375958990 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls") pod "image-registry-dfb4b6887-t9bg7" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0") : secret "image-registry-tls" not found
Sep 11 11:52:33.437624 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.437589    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bw87x"]
Sep 11 11:52:33.440993 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.440969    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.443028 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.443007    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-685qw\""
Sep 11 11:52:33.443151 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.443008    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 11:52:33.443151 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.443077    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 11:52:33.450782 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.450753    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bw87x"]
Sep 11 11:52:33.580434 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.580389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c6f3f60-4cd7-4811-83d0-5d3af4659599-data-volume\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.580599 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.580470    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcmb5\" (UniqueName: \"kubernetes.io/projected/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-api-access-vcmb5\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.580599 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.580537    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c6f3f60-4cd7-4811-83d0-5d3af4659599-crio-socket\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.580599 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.580573    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.580770 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.580714    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.662720 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.662685    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm"]
Sep 11 11:52:33.666012 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.665983    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm"
Sep 11 11:52:33.668048 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.668024    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 11:52:33.668170 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.668025    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 11:52:33.668170 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.668144    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-8dsjl\""
Sep 11 11:52:33.673604 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.673585    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm"]
Sep 11 11:52:33.681536 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.681514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c6f3f60-4cd7-4811-83d0-5d3af4659599-data-volume\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.681625 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.681550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcmb5\" (UniqueName: \"kubernetes.io/projected/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-api-access-vcmb5\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.681723 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.681698    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c6f3f60-4cd7-4811-83d0-5d3af4659599-crio-socket\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.681797 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.681762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.681797 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.681771    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c6f3f60-4cd7-4811-83d0-5d3af4659599-crio-socket\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.681897 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.681834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.681897 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.681873    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c6f3f60-4cd7-4811-83d0-5d3af4659599-data-volume\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.681987 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:33.681939    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:52:33.681987 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:33.681949    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:33.682079 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:33.682012    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:34.181993758 +0000 UTC m=+62.682714009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : configmap "kube-rbac-proxy" not found
Sep 11 11:52:33.682079 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:33.682033    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:34.182021453 +0000 UTC m=+62.682741712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:33.691827 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.691774    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcmb5\" (UniqueName: \"kubernetes.io/projected/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-api-access-vcmb5\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:33.782948 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.782914    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcr8x\" (UniqueName: \"kubernetes.io/projected/b9f83fe6-6ab6-4718-b9f5-496ce4992286-kube-api-access-qcr8x\") pod \"migrator-8b4d697c5-5vlzm\" (UID: \"b9f83fe6-6ab6-4718-b9f5-496ce4992286\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm"
Sep 11 11:52:33.883348 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.883300    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcr8x\" (UniqueName: \"kubernetes.io/projected/b9f83fe6-6ab6-4718-b9f5-496ce4992286-kube-api-access-qcr8x\") pod \"migrator-8b4d697c5-5vlzm\" (UID: \"b9f83fe6-6ab6-4718-b9f5-496ce4992286\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm"
Sep 11 11:52:33.890330 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.890294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcr8x\" (UniqueName: \"kubernetes.io/projected/b9f83fe6-6ab6-4718-b9f5-496ce4992286-kube-api-access-qcr8x\") pod \"migrator-8b4d697c5-5vlzm\" (UID: \"b9f83fe6-6ab6-4718-b9f5-496ce4992286\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm"
Sep 11 11:52:33.976129 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:33.976047    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm"
Sep 11 11:52:34.185736 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:34.185701    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:34.185914 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:34.185765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:34.185914 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:34.185862    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:52:34.185914 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:34.185867    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:34.185914 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:34.185915    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:35.185900144 +0000 UTC m=+63.686620390 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : configmap "kube-rbac-proxy" not found
Sep 11 11:52:34.186044 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:34.185928    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:35.185922006 +0000 UTC m=+63.686642251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:34.244993 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:34.244907    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm"]
Sep 11 11:52:34.248920 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:52:34.248889    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f83fe6_6ab6_4718_b9f5_496ce4992286.slice/crio-8348bfbb634e7f19a0f5bbd7ce52efdc223338a995e756cab9f4079168e36a92 WatchSource:0}: Error finding container 8348bfbb634e7f19a0f5bbd7ce52efdc223338a995e756cab9f4079168e36a92: Status 404 returned error can't find the container with id 8348bfbb634e7f19a0f5bbd7ce52efdc223338a995e756cab9f4079168e36a92
Sep 11 11:52:34.324900 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:34.324867    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm" event={"ID":"b9f83fe6-6ab6-4718-b9f5-496ce4992286","Type":"ContainerStarted","Data":"8348bfbb634e7f19a0f5bbd7ce52efdc223338a995e756cab9f4079168e36a92"}
Sep 11 11:52:35.193702 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:35.193652    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:35.194088 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:35.193775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:35.194088 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:35.193796    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:52:35.194088 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:35.193876    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:35.194088 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:35.193882    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:37.193860593 +0000 UTC m=+65.694580842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : configmap "kube-rbac-proxy" not found
Sep 11 11:52:35.194088 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:35.193938    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:37.193922398 +0000 UTC m=+65.694642661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:35.592509 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:35.592487    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xtfmj_86e098b8-e73f-43c1-bfe3-242bd950b4bd/dns-node-resolver/0.log"
Sep 11 11:52:36.332661 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:36.332623    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm" event={"ID":"b9f83fe6-6ab6-4718-b9f5-496ce4992286","Type":"ContainerStarted","Data":"df9aaaa1199794dd8daeab885e13b44102bbd1c6cc78f3003ddccc93655add41"}
Sep 11 11:52:36.333034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:36.332685    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm" event={"ID":"b9f83fe6-6ab6-4718-b9f5-496ce4992286","Type":"ContainerStarted","Data":"157d3a18c48118d2cedc9c39ce28d9c48f6b48b730769a9b0c008ecd472ef12a"}
Sep 11 11:52:36.347850 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:36.347804    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5vlzm" podStartSLOduration=2.00508136 podStartE2EDuration="3.347789369s" totalImagesPullingTime="1.342708009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:36.347241568 +0000 UTC m=+64.847961837" watchObservedRunningTime="2026-09-11 11:52:36.347789369 +0000 UTC m=+64.848509637"
Sep 11 11:52:36.393580 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:36.393555    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lv7nr_1e028310-ec98-4645-ad5c-48db2103260d/node-ca/0.log"
Sep 11 11:52:36.805601 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:36.805567    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:36.805777 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:36.805690    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:52:36.805777 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:36.805744    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert podName:b669f014-1829-47a8-9d3a-bbb01d4d6eb6 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:44.80573193 +0000 UTC m=+73.306452181 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-zq4mw" (UID: "b669f014-1829-47a8-9d3a-bbb01d4d6eb6") : secret "networking-console-plugin-cert" not found
Sep 11 11:52:36.906282 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:36.906238    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:36.906480 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:36.906422    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 11:52:36.906480 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:36.906445    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfb4b6887-t9bg7: secret "image-registry-tls" not found
Sep 11 11:52:36.906559 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:36.906499    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls podName:e4a02f19-7ae9-4d6c-850e-da97d5e406b0 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:44.906484863 +0000 UTC m=+73.407205113 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls") pod "image-registry-dfb4b6887-t9bg7" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0") : secret "image-registry-tls" not found
Sep 11 11:52:37.192613 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.192586    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d5cxg_1d546130-4201-41ee-ba52-d2f1d1b9e205/serve-healthcheck-canary/0.log"
Sep 11 11:52:37.208937 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.208893    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:37.209098 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.208955    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:37.209098 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:37.209058    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:37.209168 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:37.209127    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:41.20911266 +0000 UTC m=+69.709832911 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:37.209168 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:37.209059    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:52:37.209236 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:37.209194    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:41.20918326 +0000 UTC m=+69.709903511 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : configmap "kube-rbac-proxy" not found
Sep 11 11:52:37.714162 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.714117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:52:37.714636 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:37.714275    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 11:52:37.714636 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:37.714358    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls podName:28c2284e-89be-4c9f-a804-e1a2373a8f92 nodeName:}" failed. No retries permitted until 2026-09-11 11:53:09.714338111 +0000 UTC m=+98.215058362 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls") pod "dns-default-jsxzt" (UID: "28c2284e-89be-4c9f-a804-e1a2373a8f92") : secret "dns-default-metrics-tls" not found
Sep 11 11:52:37.814833 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.814794    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:52:37.814999 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.814850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:52:37.817016 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.817000    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 11:52:37.817125 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.817106    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 11:52:37.825489 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:37.825472    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 11:52:37.825553 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:37.825524    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs podName:142371cb-79c5-44b6-9e7d-9c01359b29f4 nodeName:}" failed. No retries permitted until 2026-09-11 11:53:41.825510262 +0000 UTC m=+130.326230508 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs") pod "network-metrics-daemon-w98wm" (UID: "142371cb-79c5-44b6-9e7d-9c01359b29f4") : secret "metrics-daemon-secret" not found
Sep 11 11:52:37.827709 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.827679    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e693b077-566f-41ce-b702-f9ea0084ed60-original-pull-secret\") pod \"global-pull-secret-syncer-fqcqr\" (UID: \"e693b077-566f-41ce-b702-f9ea0084ed60\") " pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:52:37.834569 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.834549    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fqcqr"
Sep 11 11:52:37.915793 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.915753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4vb\" (UniqueName: \"kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb\") pod \"network-check-target-t8djg\" (UID: \"1a71dff7-c64a-402f-ac3f-a0c2bb078721\") " pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:52:37.918196 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.918174    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 11:52:37.928359 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.928336    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 11:52:37.938883 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:37.938848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b4vb\" (UniqueName: \"kubernetes.io/projected/1a71dff7-c64a-402f-ac3f-a0c2bb078721-kube-api-access-4b4vb\") pod \"network-check-target-t8djg\" (UID: \"1a71dff7-c64a-402f-ac3f-a0c2bb078721\") " pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:52:38.099811 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:38.099740    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fqcqr"]
Sep 11 11:52:38.102995 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:52:38.102968    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode693b077_566f_41ce_b702_f9ea0084ed60.slice/crio-946b1e8ff5916341b420debdc46bf893f85ef6ee223f20f2ae7c5738dd9a19f3 WatchSource:0}: Error finding container 946b1e8ff5916341b420debdc46bf893f85ef6ee223f20f2ae7c5738dd9a19f3: Status 404 returned error can't find the container with id 946b1e8ff5916341b420debdc46bf893f85ef6ee223f20f2ae7c5738dd9a19f3
Sep 11 11:52:38.142227 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:38.142191    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f8bwk\""
Sep 11 11:52:38.150788 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:38.150767    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:52:38.338289 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:38.338255    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fqcqr" event={"ID":"e693b077-566f-41ce-b702-f9ea0084ed60","Type":"ContainerStarted","Data":"946b1e8ff5916341b420debdc46bf893f85ef6ee223f20f2ae7c5738dd9a19f3"}
Sep 11 11:52:38.417839 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:38.417771    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t8djg"]
Sep 11 11:52:38.421053 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:52:38.421031    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a71dff7_c64a_402f_ac3f_a0c2bb078721.slice/crio-40c4e5063786c90a013f7b203aef9d0655bd73484eba7be8e58fa7e783bb8d40 WatchSource:0}: Error finding container 40c4e5063786c90a013f7b203aef9d0655bd73484eba7be8e58fa7e783bb8d40: Status 404 returned error can't find the container with id 40c4e5063786c90a013f7b203aef9d0655bd73484eba7be8e58fa7e783bb8d40
Sep 11 11:52:39.343626 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:39.343587    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t8djg" event={"ID":"1a71dff7-c64a-402f-ac3f-a0c2bb078721","Type":"ContainerStarted","Data":"40c4e5063786c90a013f7b203aef9d0655bd73484eba7be8e58fa7e783bb8d40"}
Sep 11 11:52:41.245016 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:41.244957    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:41.245508 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:41.245057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:41.245508 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:41.245148    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 11:52:41.245508 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:41.245227    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:49.245204335 +0000 UTC m=+77.745924604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : configmap "kube-rbac-proxy" not found
Sep 11 11:52:41.245508 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:41.245146    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:41.245508 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:41.245334    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:52:49.245290876 +0000 UTC m=+77.746011144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:43.358545 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:43.358439    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fqcqr" event={"ID":"e693b077-566f-41ce-b702-f9ea0084ed60","Type":"ContainerStarted","Data":"e54fb6f8b67e7e8050c78f0a2177b0cd316a56d185a62e8e15f73fd525c15a9d"}
Sep 11 11:52:43.359959 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:43.359932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t8djg" event={"ID":"1a71dff7-c64a-402f-ac3f-a0c2bb078721","Type":"ContainerStarted","Data":"6d2c8cdee3edc8cd10c1c9dc858a5f00b1b23484801885241b5d2276e955ce1a"}
Sep 11 11:52:43.360081 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:43.360055    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:52:43.372935 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:43.372889    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fqcqr" podStartSLOduration=66.536603829 podStartE2EDuration="1m11.37287675s" totalImagesPullingTime="4.836272921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:43.371858632 +0000 UTC m=+71.872578901" watchObservedRunningTime="2026-09-11 11:52:43.37287675 +0000 UTC m=+71.873597017"
Sep 11 11:52:43.387129 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:43.387080    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-t8djg" podStartSLOduration=66.867053765 podStartE2EDuration="1m11.387066869s" totalImagesPullingTime="4.520013104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:43.386327805 +0000 UTC m=+71.887048070" watchObservedRunningTime="2026-09-11 11:52:43.387066869 +0000 UTC m=+71.887787128"
Sep 11 11:52:44.874899 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:44.874857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:52:44.875333 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:44.875009    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 11:52:44.875333 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:44.875090    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert podName:b669f014-1829-47a8-9d3a-bbb01d4d6eb6 nodeName:}" failed. No retries permitted until 2026-09-11 11:53:00.875074757 +0000 UTC m=+89.375795019 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-zq4mw" (UID: "b669f014-1829-47a8-9d3a-bbb01d4d6eb6") : secret "networking-console-plugin-cert" not found
Sep 11 11:52:44.975529 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:44.975491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls\") pod \"image-registry-dfb4b6887-t9bg7\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") " pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:44.975679 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:44.975641    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 11:52:44.975679 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:44.975661    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfb4b6887-t9bg7: secret "image-registry-tls" not found
Sep 11 11:52:44.975753 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:44.975726    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls podName:e4a02f19-7ae9-4d6c-850e-da97d5e406b0 nodeName:}" failed. No retries permitted until 2026-09-11 11:53:00.975711048 +0000 UTC m=+89.476431295 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls") pod "image-registry-dfb4b6887-t9bg7" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0") : secret "image-registry-tls" not found
Sep 11 11:52:49.310920 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:49.310873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:49.311440 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:49.310988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:49.311440 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:49.311116    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:49.311440 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:49.311189    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls podName:0c6f3f60-4cd7-4811-83d0-5d3af4659599 nodeName:}" failed. No retries permitted until 2026-09-11 11:53:05.311168624 +0000 UTC m=+93.811888885 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bw87x" (UID: "0c6f3f60-4cd7-4811-83d0-5d3af4659599") : secret "insights-runtime-extractor-tls" not found
Sep 11 11:52:49.311440 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:49.311402    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c6f3f60-4cd7-4811-83d0-5d3af4659599-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:52:55.494543 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.494398    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"]
Sep 11 11:52:55.498771 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.498747    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"
Sep 11 11:52:55.501903 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.501878    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vvscd\""
Sep 11 11:52:55.502036 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.501912    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 11 11:52:55.502036 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.501939    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 11:52:55.502036 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.501912    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 11:52:55.502036 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.501914    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 11:52:55.502968 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.502948    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dfb4b6887-t9bg7"]
Sep 11 11:52:55.503163 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:52:55.503145    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7" podUID="e4a02f19-7ae9-4d6c-850e-da97d5e406b0"
Sep 11 11:52:55.504920 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.504898    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"]
Sep 11 11:52:55.536989 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.536959    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f75696d6b-4sscr"]
Sep 11 11:52:55.539897 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.539877    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.550583 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.550561    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f75696d6b-4sscr"]
Sep 11 11:52:55.557793 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.557767    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lvd8\" (UniqueName: \"kubernetes.io/projected/18c15550-ad39-4a0b-9dfd-d6f179b2d41a-kube-api-access-4lvd8\") pod \"managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c\" (UID: \"18c15550-ad39-4a0b-9dfd-d6f179b2d41a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"
Sep 11 11:52:55.557910 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.557892    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18c15550-ad39-4a0b-9dfd-d6f179b2d41a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c\" (UID: \"18c15550-ad39-4a0b-9dfd-d6f179b2d41a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"
Sep 11 11:52:55.659024 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.658984    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd59708e-fba7-4384-9bbb-c0147c65eb81-trusted-ca\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.659191 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.659038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18c15550-ad39-4a0b-9dfd-d6f179b2d41a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c\" (UID: \"18c15550-ad39-4a0b-9dfd-d6f179b2d41a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"
Sep 11 11:52:55.659191 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.659066    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd59708e-fba7-4384-9bbb-c0147c65eb81-bound-sa-token\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.659191 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.659127    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd59708e-fba7-4384-9bbb-c0147c65eb81-installation-pull-secrets\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.659191 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.659175    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd59708e-fba7-4384-9bbb-c0147c65eb81-ca-trust-extracted\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.659351 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.659205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lvd8\" (UniqueName: \"kubernetes.io/projected/18c15550-ad39-4a0b-9dfd-d6f179b2d41a-kube-api-access-4lvd8\") pod \"managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c\" (UID: \"18c15550-ad39-4a0b-9dfd-d6f179b2d41a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"
Sep 11 11:52:55.659351 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.659250    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvfrn\" (UniqueName: \"kubernetes.io/projected/cd59708e-fba7-4384-9bbb-c0147c65eb81-kube-api-access-rvfrn\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.659414 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.659391    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd59708e-fba7-4384-9bbb-c0147c65eb81-image-registry-private-configuration\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.659470 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.659428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd59708e-fba7-4384-9bbb-c0147c65eb81-registry-tls\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.659525 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.659508    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd59708e-fba7-4384-9bbb-c0147c65eb81-registry-certificates\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.661665 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.661646    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18c15550-ad39-4a0b-9dfd-d6f179b2d41a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c\" (UID: \"18c15550-ad39-4a0b-9dfd-d6f179b2d41a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"
Sep 11 11:52:55.667194 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.667171    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lvd8\" (UniqueName: \"kubernetes.io/projected/18c15550-ad39-4a0b-9dfd-d6f179b2d41a-kube-api-access-4lvd8\") pod \"managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c\" (UID: \"18c15550-ad39-4a0b-9dfd-d6f179b2d41a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"
Sep 11 11:52:55.760602 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.760513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd59708e-fba7-4384-9bbb-c0147c65eb81-trusted-ca\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.760602 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.760554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd59708e-fba7-4384-9bbb-c0147c65eb81-bound-sa-token\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.760818 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.760609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd59708e-fba7-4384-9bbb-c0147c65eb81-installation-pull-secrets\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.760818 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.760642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd59708e-fba7-4384-9bbb-c0147c65eb81-ca-trust-extracted\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.760818 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.760670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvfrn\" (UniqueName: \"kubernetes.io/projected/cd59708e-fba7-4384-9bbb-c0147c65eb81-kube-api-access-rvfrn\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.760818 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.760711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd59708e-fba7-4384-9bbb-c0147c65eb81-image-registry-private-configuration\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.760818 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.760752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd59708e-fba7-4384-9bbb-c0147c65eb81-registry-tls\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.760818 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.760787    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd59708e-fba7-4384-9bbb-c0147c65eb81-registry-certificates\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.761177 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.761150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd59708e-fba7-4384-9bbb-c0147c65eb81-ca-trust-extracted\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.761530 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.761511    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd59708e-fba7-4384-9bbb-c0147c65eb81-trusted-ca\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.761658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.761559    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd59708e-fba7-4384-9bbb-c0147c65eb81-registry-certificates\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.763230 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.763202    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd59708e-fba7-4384-9bbb-c0147c65eb81-image-registry-private-configuration\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.763359 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.763328    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd59708e-fba7-4384-9bbb-c0147c65eb81-registry-tls\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.763420 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.763357    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd59708e-fba7-4384-9bbb-c0147c65eb81-installation-pull-secrets\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.767844 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.767807    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvfrn\" (UniqueName: \"kubernetes.io/projected/cd59708e-fba7-4384-9bbb-c0147c65eb81-kube-api-access-rvfrn\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.768797 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.768777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd59708e-fba7-4384-9bbb-c0147c65eb81-bound-sa-token\") pod \"image-registry-6f75696d6b-4sscr\" (UID: \"cd59708e-fba7-4384-9bbb-c0147c65eb81\") " pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:55.820788 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.820750    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"
Sep 11 11:52:55.850152 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.850127    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6j76c\""
Sep 11 11:52:55.858629 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:55.858598    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:56.100788 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.100689    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c"]
Sep 11 11:52:56.104628 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:52:56.104598    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18c15550_ad39_4a0b_9dfd_d6f179b2d41a.slice/crio-8deca3c3ff68d8c2c57704f9b556e1c9cd897f58e0de5589ee123add2ced4b52 WatchSource:0}: Error finding container 8deca3c3ff68d8c2c57704f9b556e1c9cd897f58e0de5589ee123add2ced4b52: Status 404 returned error can't find the container with id 8deca3c3ff68d8c2c57704f9b556e1c9cd897f58e0de5589ee123add2ced4b52
Sep 11 11:52:56.135825 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.135794    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f75696d6b-4sscr"]
Sep 11 11:52:56.138585 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:52:56.138563    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd59708e_fba7_4384_9bbb_c0147c65eb81.slice/crio-31c4b36e234c40b7de8d419b96a45795add5b10e37b69f9024e29722eea728dc WatchSource:0}: Error finding container 31c4b36e234c40b7de8d419b96a45795add5b10e37b69f9024e29722eea728dc: Status 404 returned error can't find the container with id 31c4b36e234c40b7de8d419b96a45795add5b10e37b69f9024e29722eea728dc
Sep 11 11:52:56.403981 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.403892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c" event={"ID":"18c15550-ad39-4a0b-9dfd-d6f179b2d41a","Type":"ContainerStarted","Data":"8deca3c3ff68d8c2c57704f9b556e1c9cd897f58e0de5589ee123add2ced4b52"}
Sep 11 11:52:56.405474 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.405442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f75696d6b-4sscr" event={"ID":"cd59708e-fba7-4384-9bbb-c0147c65eb81","Type":"ContainerStarted","Data":"3d7439ba6a34f2a9321b733a4e8c5d7fde3caee68a8a7ce5b8aff194c200f7d0"}
Sep 11 11:52:56.405618 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.405493    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f75696d6b-4sscr" event={"ID":"cd59708e-fba7-4384-9bbb-c0147c65eb81","Type":"ContainerStarted","Data":"31c4b36e234c40b7de8d419b96a45795add5b10e37b69f9024e29722eea728dc"}
Sep 11 11:52:56.405618 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.405454    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:56.405618 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.405543    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:52:56.415929 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.415904    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:56.426318 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.426101    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f75696d6b-4sscr" podStartSLOduration=1.426085214 podStartE2EDuration="1.426085214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:56.425488744 +0000 UTC m=+84.926209011" watchObservedRunningTime="2026-09-11 11:52:56.426085214 +0000 UTC m=+84.926805482"
Sep 11 11:52:56.466816 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.466781    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-image-registry-private-configuration\") pod \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") "
Sep 11 11:52:56.466816 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.466837    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-certificates\") pod \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") "
Sep 11 11:52:56.467084 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.466876    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-ca-trust-extracted\") pod \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") "
Sep 11 11:52:56.467158 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.467092    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cl98\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-kube-api-access-5cl98\") pod \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") "
Sep 11 11:52:56.467158 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.467141    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e4a02f19-7ae9-4d6c-850e-da97d5e406b0" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:52:56.467273 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.467161    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-bound-sa-token\") pod \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") "
Sep 11 11:52:56.467273 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.467207    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-installation-pull-secrets\") pod \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") "
Sep 11 11:52:56.467404 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.467287    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e4a02f19-7ae9-4d6c-850e-da97d5e406b0" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:56.467725 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.467696    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e4a02f19-7ae9-4d6c-850e-da97d5e406b0" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:56.467866 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.467298    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-trusted-ca\") pod \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\" (UID: \"e4a02f19-7ae9-4d6c-850e-da97d5e406b0\") "
Sep 11 11:52:56.468920 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.468227    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-certificates\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:52:56.468920 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.468253    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-ca-trust-extracted\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:52:56.468920 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.468270    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-trusted-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:52:56.469470 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.469444    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e4a02f19-7ae9-4d6c-850e-da97d5e406b0" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:56.469597 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.469568    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e4a02f19-7ae9-4d6c-850e-da97d5e406b0" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:56.469669 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.469621    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e4a02f19-7ae9-4d6c-850e-da97d5e406b0" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:56.469669 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.469652    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-kube-api-access-5cl98" (OuterVolumeSpecName: "kube-api-access-5cl98") pod "e4a02f19-7ae9-4d6c-850e-da97d5e406b0" (UID: "e4a02f19-7ae9-4d6c-850e-da97d5e406b0"). InnerVolumeSpecName "kube-api-access-5cl98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:56.569491 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.569457    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cl98\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-kube-api-access-5cl98\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:52:56.569491 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.569485    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-bound-sa-token\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:52:56.569491 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.569494    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-installation-pull-secrets\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:52:56.569491 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:56.569503    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-image-registry-private-configuration\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:52:57.408396 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:57.408364    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dfb4b6887-t9bg7"
Sep 11 11:52:57.438043 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:57.437998    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dfb4b6887-t9bg7"]
Sep 11 11:52:57.441404 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:57.441380    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-dfb4b6887-t9bg7"]
Sep 11 11:52:57.577044 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:57.576977    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a02f19-7ae9-4d6c-850e-da97d5e406b0-registry-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:52:58.023399 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:58.023361    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4a02f19-7ae9-4d6c-850e-da97d5e406b0" path="/var/lib/kubelet/pods/e4a02f19-7ae9-4d6c-850e-da97d5e406b0/volumes"
Sep 11 11:52:59.416988 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:59.416897    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c" event={"ID":"18c15550-ad39-4a0b-9dfd-d6f179b2d41a","Type":"ContainerStarted","Data":"2d14da00aed1788be1a2395856ba37e0f77a0645a48b1285c72e11119ca5d7b8"}
Sep 11 11:52:59.429990 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:52:59.429937    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df9b8d9bd-pnt6c" podStartSLOduration=1.370495255 podStartE2EDuration="4.429920709s" totalImagesPullingTime="3.059425454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:59.429583425 +0000 UTC m=+87.930303694" watchObservedRunningTime="2026-09-11 11:52:59.429920709 +0000 UTC m=+87.930640977"
Sep 11 11:53:00.906024 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:00.905986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:53:00.908389 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:00.908364    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b669f014-1829-47a8-9d3a-bbb01d4d6eb6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-zq4mw\" (UID: \"b669f014-1829-47a8-9d3a-bbb01d4d6eb6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:53:01.171696 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:01.171667    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bp9zp\""
Sep 11 11:53:01.180245 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:01.180226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"
Sep 11 11:53:01.458390 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:01.458312    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw"]
Sep 11 11:53:01.462724 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:01.462692    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb669f014_1829_47a8_9d3a_bbb01d4d6eb6.slice/crio-d45599b1f0a7ac1dc8029ed50f98e0a724be5887400f54be4a1f9487609be510 WatchSource:0}: Error finding container d45599b1f0a7ac1dc8029ed50f98e0a724be5887400f54be4a1f9487609be510: Status 404 returned error can't find the container with id d45599b1f0a7ac1dc8029ed50f98e0a724be5887400f54be4a1f9487609be510
Sep 11 11:53:02.429870 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:02.429827    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw" event={"ID":"b669f014-1829-47a8-9d3a-bbb01d4d6eb6","Type":"ContainerStarted","Data":"d45599b1f0a7ac1dc8029ed50f98e0a724be5887400f54be4a1f9487609be510"}
Sep 11 11:53:03.434319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:03.434274    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw" event={"ID":"b669f014-1829-47a8-9d3a-bbb01d4d6eb6","Type":"ContainerStarted","Data":"3fa43fd1306d0b7fcc9d3e8641c7c90051474aea00249157ab30342740c8b5c9"}
Sep 11 11:53:03.447525 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:03.447477    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-zq4mw" podStartSLOduration=32.94049132 podStartE2EDuration="34.447462849s" totalImagesPullingTime="1.506971529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:03.446706038 +0000 UTC m=+91.947426305" watchObservedRunningTime="2026-09-11 11:53:03.447462849 +0000 UTC m=+91.948183117"
Sep 11 11:53:04.081634 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:04.081595    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz"]
Sep 11 11:53:04.084857 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:04.084833    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz"
Sep 11 11:53:04.086835 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:04.086814    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 11:53:04.086932 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:04.086881    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-j7v7w\""
Sep 11 11:53:04.097253 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:04.097229    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz"]
Sep 11 11:53:04.232074 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:04.232038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5bf9ef2a-e8b5-49f8-8e5f-658d828e1a44-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-kppjz\" (UID: \"5bf9ef2a-e8b5-49f8-8e5f-658d828e1a44\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz"
Sep 11 11:53:04.333429 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:04.333328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5bf9ef2a-e8b5-49f8-8e5f-658d828e1a44-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-kppjz\" (UID: \"5bf9ef2a-e8b5-49f8-8e5f-658d828e1a44\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz"
Sep 11 11:53:04.335796 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:04.335774    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5bf9ef2a-e8b5-49f8-8e5f-658d828e1a44-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-kppjz\" (UID: \"5bf9ef2a-e8b5-49f8-8e5f-658d828e1a44\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz"
Sep 11 11:53:04.393771 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:04.393733    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz"
Sep 11 11:53:04.664986 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:04.664897    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz"]
Sep 11 11:53:04.668603 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:04.668573    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf9ef2a_e8b5_49f8_8e5f_658d828e1a44.slice/crio-e30368ebf4cbae9f20d5129517169d6bdf6a9140bc5239d7ff4f637a0fef223c WatchSource:0}: Error finding container e30368ebf4cbae9f20d5129517169d6bdf6a9140bc5239d7ff4f637a0fef223c: Status 404 returned error can't find the container with id e30368ebf4cbae9f20d5129517169d6bdf6a9140bc5239d7ff4f637a0fef223c
Sep 11 11:53:05.341680 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:05.341639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:53:05.346379 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:05.346348    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c6f3f60-4cd7-4811-83d0-5d3af4659599-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bw87x\" (UID: \"0c6f3f60-4cd7-4811-83d0-5d3af4659599\") " pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:53:05.442383 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:05.442340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz" event={"ID":"5bf9ef2a-e8b5-49f8-8e5f-658d828e1a44","Type":"ContainerStarted","Data":"e30368ebf4cbae9f20d5129517169d6bdf6a9140bc5239d7ff4f637a0fef223c"}
Sep 11 11:53:05.551085 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:05.551045    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bw87x"
Sep 11 11:53:05.822527 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:05.822488    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bw87x"]
Sep 11 11:53:05.965946 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:05.965912    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c6f3f60_4cd7_4811_83d0_5d3af4659599.slice/crio-4cf163aa2d4bcd3988021e274985044512bc1273411a98395ffd78c0af095d06 WatchSource:0}: Error finding container 4cf163aa2d4bcd3988021e274985044512bc1273411a98395ffd78c0af095d06: Status 404 returned error can't find the container with id 4cf163aa2d4bcd3988021e274985044512bc1273411a98395ffd78c0af095d06
Sep 11 11:53:06.446867 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:06.446833    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz" event={"ID":"5bf9ef2a-e8b5-49f8-8e5f-658d828e1a44","Type":"ContainerStarted","Data":"67ec56c44617920ccad9390fa6a7765d139906346cf662d937ea6a368c0f69b7"}
Sep 11 11:53:06.447043 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:06.447027    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz"
Sep 11 11:53:06.448487 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:06.448464    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bw87x" event={"ID":"0c6f3f60-4cd7-4811-83d0-5d3af4659599","Type":"ContainerStarted","Data":"7c689302f84d9ef8ab25627766594967d766a1f9831ed3e41ee0580507e64fc7"}
Sep 11 11:53:06.448688 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:06.448512    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bw87x" event={"ID":"0c6f3f60-4cd7-4811-83d0-5d3af4659599","Type":"ContainerStarted","Data":"4cf163aa2d4bcd3988021e274985044512bc1273411a98395ffd78c0af095d06"}
Sep 11 11:53:06.451837 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:06.451818    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz"
Sep 11 11:53:06.463657 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:06.463617    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-kppjz" podStartSLOduration=1.112115624 podStartE2EDuration="2.463599975s" totalImagesPullingTime="1.351484351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:06.462833763 +0000 UTC m=+94.963554032" watchObservedRunningTime="2026-09-11 11:53:06.463599975 +0000 UTC m=+94.964320242"
Sep 11 11:53:07.137947 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.137901    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"]
Sep 11 11:53:07.140852 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.140835    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.143347 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.143327    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 11 11:53:07.143465 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.143330    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 11:53:07.143465 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.143368    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 11 11:53:07.143465 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.143355    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-l7582\""
Sep 11 11:53:07.143465 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.143355    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 11:53:07.143655 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.143628    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 11:53:07.151340 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.151316    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"]
Sep 11 11:53:07.258052 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.257961    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5026325a-81a2-44b8-a246-2d650d5852b7-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.258052 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.257997    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5026325a-81a2-44b8-a246-2d650d5852b7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.258237 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.258082    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5026325a-81a2-44b8-a246-2d650d5852b7-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.258237 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.258123    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcmf2\" (UniqueName: \"kubernetes.io/projected/5026325a-81a2-44b8-a246-2d650d5852b7-kube-api-access-jcmf2\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.359280 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.359234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5026325a-81a2-44b8-a246-2d650d5852b7-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.359280 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.359282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5026325a-81a2-44b8-a246-2d650d5852b7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.359525 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.359394    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5026325a-81a2-44b8-a246-2d650d5852b7-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.359525 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.359442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcmf2\" (UniqueName: \"kubernetes.io/projected/5026325a-81a2-44b8-a246-2d650d5852b7-kube-api-access-jcmf2\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.360140 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.360114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5026325a-81a2-44b8-a246-2d650d5852b7-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.361797 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.361775    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5026325a-81a2-44b8-a246-2d650d5852b7-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.361867 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.361823    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5026325a-81a2-44b8-a246-2d650d5852b7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.366454 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.366430    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcmf2\" (UniqueName: \"kubernetes.io/projected/5026325a-81a2-44b8-a246-2d650d5852b7-kube-api-access-jcmf2\") pod \"prometheus-operator-68bcd7f987-lwzhv\" (UID: \"5026325a-81a2-44b8-a246-2d650d5852b7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.449520 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.449486    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"
Sep 11 11:53:07.453336 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.453314    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bw87x" event={"ID":"0c6f3f60-4cd7-4811-83d0-5d3af4659599","Type":"ContainerStarted","Data":"bedbfd7109a200d8797aef1d47d495033eaec00f66050640f28fa3ff81006276"}
Sep 11 11:53:07.717616 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:07.717581    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv"]
Sep 11 11:53:07.721180 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:07.721152    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5026325a_81a2_44b8_a246_2d650d5852b7.slice/crio-117d5512a6ed881d9c82c4b5c486a842990bf89f3c3c5fbb43259b4f5ea16857 WatchSource:0}: Error finding container 117d5512a6ed881d9c82c4b5c486a842990bf89f3c3c5fbb43259b4f5ea16857: Status 404 returned error can't find the container with id 117d5512a6ed881d9c82c4b5c486a842990bf89f3c3c5fbb43259b4f5ea16857
Sep 11 11:53:08.458095 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:08.458064    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv" event={"ID":"5026325a-81a2-44b8-a246-2d650d5852b7","Type":"ContainerStarted","Data":"117d5512a6ed881d9c82c4b5c486a842990bf89f3c3c5fbb43259b4f5ea16857"}
Sep 11 11:53:09.465189 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:09.465148    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv" event={"ID":"5026325a-81a2-44b8-a246-2d650d5852b7","Type":"ContainerStarted","Data":"fbe36f60db29427ed0d183ff86d456e8a4b5b9573b6e25e9c93bcf61dda17ea8"}
Sep 11 11:53:09.465651 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:09.465239    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv" event={"ID":"5026325a-81a2-44b8-a246-2d650d5852b7","Type":"ContainerStarted","Data":"a2b9d37f88b6e22777ba713841bba78a15f0f6629f5a2d29f270a757f224219a"}
Sep 11 11:53:09.467079 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:09.467055    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bw87x" event={"ID":"0c6f3f60-4cd7-4811-83d0-5d3af4659599","Type":"ContainerStarted","Data":"226d842aab0c80cab820b556620c5bd1f229cef5bb01b0cc6f840db6ffa25583"}
Sep 11 11:53:09.481031 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:09.480979    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-lwzhv" podStartSLOduration=1.035110926 podStartE2EDuration="2.480964614s" totalImagesPullingTime="1.445853688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:09.479014359 +0000 UTC m=+97.979734628" watchObservedRunningTime="2026-09-11 11:53:09.480964614 +0000 UTC m=+97.981684882"
Sep 11 11:53:09.495205 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:09.495156    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bw87x" podStartSLOduration=34.154068089 podStartE2EDuration="36.495141179s" totalImagesPullingTime="2.34107309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:52:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:09.494172287 +0000 UTC m=+97.994892556" watchObservedRunningTime="2026-09-11 11:53:09.495141179 +0000 UTC m=+97.995861486"
Sep 11 11:53:09.782703 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:09.782617    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:53:09.784866 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:09.784847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/28c2284e-89be-4c9f-a804-e1a2373a8f92-metrics-tls\") pod \"dns-default-jsxzt\" (UID: \"28c2284e-89be-4c9f-a804-e1a2373a8f92\") " pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:53:09.815874 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:09.815847    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mvnq6\""
Sep 11 11:53:09.824425 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:09.824407    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:53:10.095332 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:10.095240    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jsxzt"]
Sep 11 11:53:10.098530 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:10.098507    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c2284e_89be_4c9f_a804_e1a2373a8f92.slice/crio-f7ca6c4bf85b7e8140487ee9df6c91dcfb0c104268c27b871cee803f50e84b5f WatchSource:0}: Error finding container f7ca6c4bf85b7e8140487ee9df6c91dcfb0c104268c27b871cee803f50e84b5f: Status 404 returned error can't find the container with id f7ca6c4bf85b7e8140487ee9df6c91dcfb0c104268c27b871cee803f50e84b5f
Sep 11 11:53:10.471278 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:10.471242    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jsxzt" event={"ID":"28c2284e-89be-4c9f-a804-e1a2373a8f92","Type":"ContainerStarted","Data":"f7ca6c4bf85b7e8140487ee9df6c91dcfb0c104268c27b871cee803f50e84b5f"}
Sep 11 11:53:11.474451 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.474422    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"]
Sep 11 11:53:11.477816 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.477791    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.480148 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.480069    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 11:53:11.480335 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.480299    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 11:53:11.480335 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.480328    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ttfrb\""
Sep 11 11:53:11.489071 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.489048    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"]
Sep 11 11:53:11.490034 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.490013    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7xdvd"]
Sep 11 11:53:11.493417 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.493399    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.495288 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.495268    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 11:53:11.496021 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.495996    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gz4mc\""
Sep 11 11:53:11.496143 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.496081    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 11:53:11.496367 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.496349    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 11:53:11.598464 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2dcce9c-c706-4269-baf4-f1a153b9345d-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.598657 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-textfile\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.598657 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598514    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-accelerators-collector-config\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.598657 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7109f4b4-3483-45f3-9bbd-650fc0ca8280-sys\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.598657 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598639    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grkb8\" (UniqueName: \"kubernetes.io/projected/7109f4b4-3483-45f3-9bbd-650fc0ca8280-kube-api-access-grkb8\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.598841 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598689    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7109f4b4-3483-45f3-9bbd-650fc0ca8280-root\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.598841 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7109f4b4-3483-45f3-9bbd-650fc0ca8280-metrics-client-ca\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.598841 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598741    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-tls\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.598841 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598758    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l95pr\" (UniqueName: \"kubernetes.io/projected/a2dcce9c-c706-4269-baf4-f1a153b9345d-kube-api-access-l95pr\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.598841 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598784    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2dcce9c-c706-4269-baf4-f1a153b9345d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.598841 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2dcce9c-c706-4269-baf4-f1a153b9345d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.599090 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598890    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-wtmp\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.599090 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.598913    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.699757 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.699716    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7109f4b4-3483-45f3-9bbd-650fc0ca8280-root\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.699942 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.699830    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7109f4b4-3483-45f3-9bbd-650fc0ca8280-root\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.699942 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.699870    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7109f4b4-3483-45f3-9bbd-650fc0ca8280-metrics-client-ca\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.699942 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.699899    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-tls\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.699942 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.699925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l95pr\" (UniqueName: \"kubernetes.io/projected/a2dcce9c-c706-4269-baf4-f1a153b9345d-kube-api-access-l95pr\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.700118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.699958    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2dcce9c-c706-4269-baf4-f1a153b9345d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.700118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.699988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2dcce9c-c706-4269-baf4-f1a153b9345d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.700118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700037    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-wtmp\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.700118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700061    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.700118 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2dcce9c-c706-4269-baf4-f1a153b9345d-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.700767 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700145    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-textfile\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.700767 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700172    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-accelerators-collector-config\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.700767 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:53:11.700194    2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 11:53:11.700767 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7109f4b4-3483-45f3-9bbd-650fc0ca8280-sys\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.700767 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700241    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grkb8\" (UniqueName: \"kubernetes.io/projected/7109f4b4-3483-45f3-9bbd-650fc0ca8280-kube-api-access-grkb8\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.700767 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:53:11.700256    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2dcce9c-c706-4269-baf4-f1a153b9345d-openshift-state-metrics-tls podName:a2dcce9c-c706-4269-baf4-f1a153b9345d nodeName:}" failed. No retries permitted until 2026-09-11 11:53:12.200237089 +0000 UTC m=+100.700957348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a2dcce9c-c706-4269-baf4-f1a153b9345d-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-dlgkn" (UID: "a2dcce9c-c706-4269-baf4-f1a153b9345d") : secret "openshift-state-metrics-tls" not found
Sep 11 11:53:11.700767 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700259    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-wtmp\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.700767 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7109f4b4-3483-45f3-9bbd-650fc0ca8280-sys\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.700767 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700652    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7109f4b4-3483-45f3-9bbd-650fc0ca8280-metrics-client-ca\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.701293 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-textfile\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.701293 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.700882    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-accelerators-collector-config\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.701293 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.701140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2dcce9c-c706-4269-baf4-f1a153b9345d-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.702905 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.702863    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.703081 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.703059    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7109f4b4-3483-45f3-9bbd-650fc0ca8280-node-exporter-tls\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.703141 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.703059    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2dcce9c-c706-4269-baf4-f1a153b9345d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.710285 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.710257    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l95pr\" (UniqueName: \"kubernetes.io/projected/a2dcce9c-c706-4269-baf4-f1a153b9345d-kube-api-access-l95pr\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:11.711203 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.711181    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grkb8\" (UniqueName: \"kubernetes.io/projected/7109f4b4-3483-45f3-9bbd-650fc0ca8280-kube-api-access-grkb8\") pod \"node-exporter-7xdvd\" (UID: \"7109f4b4-3483-45f3-9bbd-650fc0ca8280\") " pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.804147 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:11.804058    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7xdvd"
Sep 11 11:53:11.912139 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:11.912111    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7109f4b4_3483_45f3_9bbd_650fc0ca8280.slice/crio-997072894d7de5785271443a0afa5fbaaa1cd9d3cb24a076d3b7ee4b61111e62 WatchSource:0}: Error finding container 997072894d7de5785271443a0afa5fbaaa1cd9d3cb24a076d3b7ee4b61111e62: Status 404 returned error can't find the container with id 997072894d7de5785271443a0afa5fbaaa1cd9d3cb24a076d3b7ee4b61111e62
Sep 11 11:53:12.204703 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:12.204659    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2dcce9c-c706-4269-baf4-f1a153b9345d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:12.207042 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:12.207022    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2dcce9c-c706-4269-baf4-f1a153b9345d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dlgkn\" (UID: \"a2dcce9c-c706-4269-baf4-f1a153b9345d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:12.389166 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:12.389071    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"
Sep 11 11:53:12.482921 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:12.482888    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jsxzt" event={"ID":"28c2284e-89be-4c9f-a804-e1a2373a8f92","Type":"ContainerStarted","Data":"771d3fe02f135710821fe93ab98e2266b4d55b98b45bb2c601f89493dbf578f0"}
Sep 11 11:53:12.483367 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:12.482956    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jsxzt" event={"ID":"28c2284e-89be-4c9f-a804-e1a2373a8f92","Type":"ContainerStarted","Data":"f196902c10ffc2cdc1ab90905f914de3d964652ba5debdc9fcceb758ce307bf5"}
Sep 11 11:53:12.483367 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:12.483017    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:53:12.484535 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:12.484504    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7xdvd" event={"ID":"7109f4b4-3483-45f3-9bbd-650fc0ca8280","Type":"ContainerStarted","Data":"997072894d7de5785271443a0afa5fbaaa1cd9d3cb24a076d3b7ee4b61111e62"}
Sep 11 11:53:12.500394 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:12.500335    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jsxzt" podStartSLOduration=65.64730034 podStartE2EDuration="1m7.500312692s" totalImagesPullingTime="1.853012352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:12.499928107 +0000 UTC m=+101.000648375" watchObservedRunningTime="2026-09-11 11:53:12.500312692 +0000 UTC m=+101.001032953"
Sep 11 11:53:12.667193 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:12.666905    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn"]
Sep 11 11:53:12.670092 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:12.670064    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2dcce9c_c706_4269_baf4_f1a153b9345d.slice/crio-bdd005ef64354acbe6ee290c0dbbccbc5d0dd9514fb761b05e7b612e646046c0 WatchSource:0}: Error finding container bdd005ef64354acbe6ee290c0dbbccbc5d0dd9514fb761b05e7b612e646046c0: Status 404 returned error can't find the container with id bdd005ef64354acbe6ee290c0dbbccbc5d0dd9514fb761b05e7b612e646046c0
Sep 11 11:53:13.469279 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.469242    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"]
Sep 11 11:53:13.510211 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.510183    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"]
Sep 11 11:53:13.510608 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.510257    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn" event={"ID":"a2dcce9c-c706-4269-baf4-f1a153b9345d","Type":"ContainerStarted","Data":"a4de4604e600dc0cf5efc135471dabeb7bf8d78d5ffecf8d1efad55af3464a95"}
Sep 11 11:53:13.510608 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.510297    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.510608 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.510315    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn" event={"ID":"a2dcce9c-c706-4269-baf4-f1a153b9345d","Type":"ContainerStarted","Data":"692f3926228a0f65a8683c58f7d49be8e3f8dbc22bd9a4c9bdff27b2ea0a2e7a"}
Sep 11 11:53:13.510608 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.510352    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn" event={"ID":"a2dcce9c-c706-4269-baf4-f1a153b9345d","Type":"ContainerStarted","Data":"bdd005ef64354acbe6ee290c0dbbccbc5d0dd9514fb761b05e7b612e646046c0"}
Sep 11 11:53:13.513080 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.513052    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 11:53:13.513206 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.513094    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bg3dnbkbp2mgh\""
Sep 11 11:53:13.513206 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.513155    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 11:53:13.513206 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.513174    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 11:53:13.513206 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.513100    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 11:53:13.513383 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.513216    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 11:53:13.513528 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.513514    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-gcjrc\""
Sep 11 11:53:13.617833 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.617801    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jct7z\" (UniqueName: \"kubernetes.io/projected/b8276f62-610e-4721-b276-1f648166036e-kube-api-access-jct7z\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.618038 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.617847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8276f62-610e-4721-b276-1f648166036e-metrics-client-ca\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.618038 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.617997    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.618125 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.618066    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.618125 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.618100    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.618202 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.618123    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-grpc-tls\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.618202 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.618173    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-tls\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.618269 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.618241    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.719102 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.719017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.719102 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.719055    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.719102 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.719073    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-grpc-tls\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.719408 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.719227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-tls\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.719408 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.719267    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.719408 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.719323    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jct7z\" (UniqueName: \"kubernetes.io/projected/b8276f62-610e-4721-b276-1f648166036e-kube-api-access-jct7z\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.719561 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.719470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8276f62-610e-4721-b276-1f648166036e-metrics-client-ca\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.719561 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.719534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.720419 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.720393    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8276f62-610e-4721-b276-1f648166036e-metrics-client-ca\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.721880 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.721853    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.721979 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.721891    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.722125 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.722104    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.722271 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.722252    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.722367 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.722348    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-thanos-querier-tls\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.722404 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.722368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b8276f62-610e-4721-b276-1f648166036e-secret-grpc-tls\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.726668 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.726649    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jct7z\" (UniqueName: \"kubernetes.io/projected/b8276f62-610e-4721-b276-1f648166036e-kube-api-access-jct7z\") pod \"thanos-querier-68b56d7dcb-4slw2\" (UID: \"b8276f62-610e-4721-b276-1f648166036e\") " pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:13.819867 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:13.819829    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:14.117528 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:14.117435    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"]
Sep 11 11:53:14.335174 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:14.335134    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8276f62_610e_4721_b276_1f648166036e.slice/crio-420a78cd6d18a2efd494025242f74237515bda5ff31af342911c23a2b23787ea WatchSource:0}: Error finding container 420a78cd6d18a2efd494025242f74237515bda5ff31af342911c23a2b23787ea: Status 404 returned error can't find the container with id 420a78cd6d18a2efd494025242f74237515bda5ff31af342911c23a2b23787ea
Sep 11 11:53:14.366437 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:14.366398    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-t8djg"
Sep 11 11:53:14.498459 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:14.498411    2562 generic.go:356] "Generic (PLEG): container finished" podID="7109f4b4-3483-45f3-9bbd-650fc0ca8280" containerID="4a5d2e7b8bc288b39a46ce1743740d896e9637773f1084d93f313384954c2402" exitCode=0
Sep 11 11:53:14.498620 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:14.498505    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7xdvd" event={"ID":"7109f4b4-3483-45f3-9bbd-650fc0ca8280","Type":"ContainerDied","Data":"4a5d2e7b8bc288b39a46ce1743740d896e9637773f1084d93f313384954c2402"}
Sep 11 11:53:14.499798 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:14.499776    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2" event={"ID":"b8276f62-610e-4721-b276-1f648166036e","Type":"ContainerStarted","Data":"420a78cd6d18a2efd494025242f74237515bda5ff31af342911c23a2b23787ea"}
Sep 11 11:53:15.506599 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:15.506560    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7xdvd" event={"ID":"7109f4b4-3483-45f3-9bbd-650fc0ca8280","Type":"ContainerStarted","Data":"6c1ea067d09d3f6c9f9efb6cd6f6752e22d40ae89f2b21b9b694e8bebc9b7d1d"}
Sep 11 11:53:15.506988 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:15.506618    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7xdvd" event={"ID":"7109f4b4-3483-45f3-9bbd-650fc0ca8280","Type":"ContainerStarted","Data":"20d27f4e4e842e2fa351b657387d320ac39187e3175ec3ef07940f225c61d076"}
Sep 11 11:53:15.508617 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:15.508595    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn" event={"ID":"a2dcce9c-c706-4269-baf4-f1a153b9345d","Type":"ContainerStarted","Data":"2dbefadccc45e8be460db90a3dc928b904a6e0bac4521b94b35d15ef2f27a31c"}
Sep 11 11:53:15.524359 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:15.524285    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7xdvd" podStartSLOduration=2.913874737 podStartE2EDuration="4.524265461s" totalImagesPullingTime="1.529920182s" totalInitContainerRuntime="80.470542ms" isStatefulPod=true podCreationTimestamp="2026-09-11 11:53:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:15.522592586 +0000 UTC m=+104.023312857" watchObservedRunningTime="2026-09-11 11:53:15.524265461 +0000 UTC m=+104.024985730"
Sep 11 11:53:15.543416 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:15.543363    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dlgkn" podStartSLOduration=2.999837068 podStartE2EDuration="4.54334655s" totalImagesPullingTime="1.543509482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:15.542443336 +0000 UTC m=+104.043163604" watchObservedRunningTime="2026-09-11 11:53:15.54334655 +0000 UTC m=+104.044066818"
Sep 11 11:53:15.863557 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:15.863463    2562 patch_prober.go:28] interesting pod/image-registry-6f75696d6b-4sscr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 11:53:15.863710 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:15.863547    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6f75696d6b-4sscr" podUID="cd59708e-fba7-4384-9bbb-c0147c65eb81" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 11:53:16.519086 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.519055    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2" event={"ID":"b8276f62-610e-4721-b276-1f648166036e","Type":"ContainerStarted","Data":"ee70f293f2245c8ae56026c7b3535e5ec790d76433269567c00d49ebbe5b9aab"}
Sep 11 11:53:16.519495 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.519167    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2" event={"ID":"b8276f62-610e-4721-b276-1f648166036e","Type":"ContainerStarted","Data":"87eaeda59a1d7e5fbc7f36496919661cf53b3d8af2a90c902223b088492fcbad"}
Sep 11 11:53:16.684219 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.684182    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-77c496dd8f-9k224"]
Sep 11 11:53:16.687717 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.687702    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.689689 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.689650    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 11 11:53:16.689820 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.689730    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 11 11:53:16.689820 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.689650    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 11 11:53:16.689820 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.689784    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 11 11:53:16.689973 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.689893    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lsbmq\""
Sep 11 11:53:16.689973 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.689955    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 11 11:53:16.694581 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.694560    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 11 11:53:16.699355 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.699335    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77c496dd8f-9k224"]
Sep 11 11:53:16.732153 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.732125    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d76cfd779-pttwl"]
Sep 11 11:53:16.735252 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.735237    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.737435 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.737410    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 11:53:16.737561 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.737419    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 11:53:16.737561 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.737465    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 11:53:16.737662 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.737603    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 11:53:16.737819 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.737806    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wrkjb\""
Sep 11 11:53:16.737858 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.737814    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 11:53:16.738038 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.738025    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 11:53:16.738289 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.738273    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 11:53:16.743120 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.743062    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 11:53:16.744572 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.744551    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d76cfd779-pttwl"]
Sep 11 11:53:16.848634 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848601    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c177b06d-d37c-4e70-a22b-4646b90d4e4b-metrics-client-ca\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.848634 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848639    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.848903 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848658    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-trusted-ca-bundle\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.848903 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-telemeter-client-tls\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.848903 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848716    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-federate-client-tls\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.848903 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c177b06d-d37c-4e70-a22b-4646b90d4e4b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.848903 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848798    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-secret-telemeter-client\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.848903 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848853    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-console-config\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.848903 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848893    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-oauth-serving-cert\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.849217 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848926    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6hhz\" (UniqueName: \"kubernetes.io/projected/d2238619-09b8-4913-96d2-7354d40179c6-kube-api-access-m6hhz\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.849217 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848962    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c177b06d-d37c-4e70-a22b-4646b90d4e4b-serving-certs-ca-bundle\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.849217 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.848997    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq4b9\" (UniqueName: \"kubernetes.io/projected/c177b06d-d37c-4e70-a22b-4646b90d4e4b-kube-api-access-dq4b9\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.849217 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.849042    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-oauth-config\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.849217 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.849100    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-service-ca\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.849217 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.849145    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-serving-cert\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.949927 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.949828    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-oauth-config\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.949927 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.949878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-service-ca\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.949927 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.949900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-serving-cert\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.949927 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.949921    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c177b06d-d37c-4e70-a22b-4646b90d4e4b-metrics-client-ca\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.950258 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950104    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.950258 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950147    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-trusted-ca-bundle\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.950258 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950189    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-telemeter-client-tls\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.950258 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950217    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-federate-client-tls\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.950258 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950252    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c177b06d-d37c-4e70-a22b-4646b90d4e4b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.950554 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950340    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-secret-telemeter-client\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.950554 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950387    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-console-config\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.950554 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-oauth-serving-cert\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.950554 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6hhz\" (UniqueName: \"kubernetes.io/projected/d2238619-09b8-4913-96d2-7354d40179c6-kube-api-access-m6hhz\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.950554 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c177b06d-d37c-4e70-a22b-4646b90d4e4b-serving-certs-ca-bundle\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.950554 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq4b9\" (UniqueName: \"kubernetes.io/projected/c177b06d-d37c-4e70-a22b-4646b90d4e4b-kube-api-access-dq4b9\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.950840 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950710    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-service-ca\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.950840 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.950724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c177b06d-d37c-4e70-a22b-4646b90d4e4b-metrics-client-ca\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.951122 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.951098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-trusted-ca-bundle\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.951376 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.951353    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-console-config\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.951500 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.951480    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c177b06d-d37c-4e70-a22b-4646b90d4e4b-serving-certs-ca-bundle\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.951668 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.951643    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-oauth-serving-cert\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.951951 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.951922    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c177b06d-d37c-4e70-a22b-4646b90d4e4b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.952979 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.952948    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-oauth-config\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.953206 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.953179    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-serving-cert\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.953534 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.953513    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.953936 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.953906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-secret-telemeter-client\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.954038 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.954018    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-federate-client-tls\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.954250 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.954234    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c177b06d-d37c-4e70-a22b-4646b90d4e4b-telemeter-client-tls\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.958016 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.957992    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6hhz\" (UniqueName: \"kubernetes.io/projected/d2238619-09b8-4913-96d2-7354d40179c6-kube-api-access-m6hhz\") pod \"console-7d76cfd779-pttwl\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") " pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:16.958434 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.958407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq4b9\" (UniqueName: \"kubernetes.io/projected/c177b06d-d37c-4e70-a22b-4646b90d4e4b-kube-api-access-dq4b9\") pod \"telemeter-client-77c496dd8f-9k224\" (UID: \"c177b06d-d37c-4e70-a22b-4646b90d4e4b\") " pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:16.997262 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:16.997226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224"
Sep 11 11:53:17.045800 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:17.045744    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:17.282850 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:17.282748    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77c496dd8f-9k224"]
Sep 11 11:53:17.323357 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:17.323324    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d76cfd779-pttwl"]
Sep 11 11:53:17.374787 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:17.374751    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc177b06d_d37c_4e70_a22b_4646b90d4e4b.slice/crio-f9b04ba4cfd36d11ebf0cdb81a9e76e542d267485e6d45cb9c83f9d11cd7b18d WatchSource:0}: Error finding container f9b04ba4cfd36d11ebf0cdb81a9e76e542d267485e6d45cb9c83f9d11cd7b18d: Status 404 returned error can't find the container with id f9b04ba4cfd36d11ebf0cdb81a9e76e542d267485e6d45cb9c83f9d11cd7b18d
Sep 11 11:53:17.375780 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:17.375455    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2238619_09b8_4913_96d2_7354d40179c6.slice/crio-a1344cbde5cb7e57519114ad2c78e26ed5853cbe621292b49dccdd55a756423b WatchSource:0}: Error finding container a1344cbde5cb7e57519114ad2c78e26ed5853cbe621292b49dccdd55a756423b: Status 404 returned error can't find the container with id a1344cbde5cb7e57519114ad2c78e26ed5853cbe621292b49dccdd55a756423b
Sep 11 11:53:17.412571 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:17.412544    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f75696d6b-4sscr"
Sep 11 11:53:17.527978 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:17.527919    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2" event={"ID":"b8276f62-610e-4721-b276-1f648166036e","Type":"ContainerStarted","Data":"1ba1503b9f69cf4f0a9b727e10031821284ac37f16266f3455b5a5c55e4ae2ce"}
Sep 11 11:53:17.528388 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:17.528056    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2" event={"ID":"b8276f62-610e-4721-b276-1f648166036e","Type":"ContainerStarted","Data":"717849baf249905356cfb9ab8ed701f38a8380c1c670b9e457bb8daa96254a4e"}
Sep 11 11:53:17.529358 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:17.529337    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224" event={"ID":"c177b06d-d37c-4e70-a22b-4646b90d4e4b","Type":"ContainerStarted","Data":"f9b04ba4cfd36d11ebf0cdb81a9e76e542d267485e6d45cb9c83f9d11cd7b18d"}
Sep 11 11:53:17.530813 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:17.530792    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d76cfd779-pttwl" event={"ID":"d2238619-09b8-4913-96d2-7354d40179c6","Type":"ContainerStarted","Data":"a1344cbde5cb7e57519114ad2c78e26ed5853cbe621292b49dccdd55a756423b"}
Sep 11 11:53:18.540314 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:18.540267    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2" event={"ID":"b8276f62-610e-4721-b276-1f648166036e","Type":"ContainerStarted","Data":"5420c92c7ec1222e55870e28fce0c01bbe4524d370bf4e5f490f341d8c8c1f75"}
Sep 11 11:53:18.540787 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:18.540379    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2" event={"ID":"b8276f62-610e-4721-b276-1f648166036e","Type":"ContainerStarted","Data":"6846faf2469a70a2552150d5e9ffe39b89393869cf545b082621047993a6fcb6"}
Sep 11 11:53:18.540787 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:18.540634    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:18.564835 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:18.564776    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2" podStartSLOduration=2.674512789 podStartE2EDuration="5.564753472s" totalImagesPullingTime="2.890240683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:18.562350356 +0000 UTC m=+107.063070628" watchObservedRunningTime="2026-09-11 11:53:18.564753472 +0000 UTC m=+107.065473741"
Sep 11 11:53:20.552625 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:20.552516    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d76cfd779-pttwl" event={"ID":"d2238619-09b8-4913-96d2-7354d40179c6","Type":"ContainerStarted","Data":"50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582"}
Sep 11 11:53:20.554056 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:20.554031    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224" event={"ID":"c177b06d-d37c-4e70-a22b-4646b90d4e4b","Type":"ContainerStarted","Data":"9f91f2053c5cceff1253034462df72b5bd879e648f600e7edf4e9e52df55336b"}
Sep 11 11:53:20.574208 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:20.574157    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d76cfd779-pttwl" podStartSLOduration=1.644039196 podStartE2EDuration="4.574138672s" totalImagesPullingTime="2.930099476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:20.573064901 +0000 UTC m=+109.073785170" watchObservedRunningTime="2026-09-11 11:53:20.574138672 +0000 UTC m=+109.074858939"
Sep 11 11:53:21.559146 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:21.559118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224" event={"ID":"c177b06d-d37c-4e70-a22b-4646b90d4e4b","Type":"ContainerStarted","Data":"26e1947650c6d256b01722987b123a983fdb2cf258ccda20ae90619be79ad044"}
Sep 11 11:53:22.493569 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.493533    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jsxzt"
Sep 11 11:53:22.566774 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.566734    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224" event={"ID":"c177b06d-d37c-4e70-a22b-4646b90d4e4b","Type":"ContainerStarted","Data":"16c0e613c3735b9c97219c611243d81c67add47e22fcc56282b4c6dc07bb909e"}
Sep 11 11:53:22.585708 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.585655    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-77c496dd8f-9k224" podStartSLOduration=2.577397301 podStartE2EDuration="6.585632813s" totalImagesPullingTime="4.008235512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:22.585038568 +0000 UTC m=+111.085758838" watchObservedRunningTime="2026-09-11 11:53:22.585632813 +0000 UTC m=+111.086353082"
Sep 11 11:53:22.730014 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.729974    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68d47ddfc4-rxsjk"]
Sep 11 11:53:22.733235 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.733217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.745509 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.745443    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68d47ddfc4-rxsjk"]
Sep 11 11:53:22.805685 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.805648    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-oauth-serving-cert\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.805868 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.805700    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-trusted-ca-bundle\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.805868 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.805746    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-oauth-config\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.805868 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.805849    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-service-ca\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.806025 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.805917    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-serving-cert\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.806025 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.805956    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-config\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.806025 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.805985    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wltnq\" (UniqueName: \"kubernetes.io/projected/18e11096-18b2-46b4-ad33-b2de2911bfc4-kube-api-access-wltnq\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.906730 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.906683    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-oauth-config\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.906730 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.906730    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-service-ca\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.906955 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.906759    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-serving-cert\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.906955 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.906780    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-config\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.906955 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.906797    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wltnq\" (UniqueName: \"kubernetes.io/projected/18e11096-18b2-46b4-ad33-b2de2911bfc4-kube-api-access-wltnq\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.906955 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.906832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-oauth-serving-cert\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.906955 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.906867    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-trusted-ca-bundle\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.907650 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.907613    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-service-ca\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.909329 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.908357    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-config\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.909329 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.909190    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-oauth-serving-cert\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.909721 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.909695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-oauth-config\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.910022 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.909995    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-serving-cert\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.910272 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.910250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-trusted-ca-bundle\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:22.916113 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:22.915968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wltnq\" (UniqueName: \"kubernetes.io/projected/18e11096-18b2-46b4-ad33-b2de2911bfc4-kube-api-access-wltnq\") pod \"console-68d47ddfc4-rxsjk\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") " pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:23.042736 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:23.042645    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:23.319403 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:23.319329    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68d47ddfc4-rxsjk"]
Sep 11 11:53:23.322401 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:23.322366    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e11096_18b2_46b4_ad33_b2de2911bfc4.slice/crio-f794bc565a8209317b4d5c4d902576ff3cdb41cf164514c6d94435a4a7f669ae WatchSource:0}: Error finding container f794bc565a8209317b4d5c4d902576ff3cdb41cf164514c6d94435a4a7f669ae: Status 404 returned error can't find the container with id f794bc565a8209317b4d5c4d902576ff3cdb41cf164514c6d94435a4a7f669ae
Sep 11 11:53:23.572602 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:23.572508    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68d47ddfc4-rxsjk" event={"ID":"18e11096-18b2-46b4-ad33-b2de2911bfc4","Type":"ContainerStarted","Data":"7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775"}
Sep 11 11:53:23.572602 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:23.572572    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68d47ddfc4-rxsjk" event={"ID":"18e11096-18b2-46b4-ad33-b2de2911bfc4","Type":"ContainerStarted","Data":"f794bc565a8209317b4d5c4d902576ff3cdb41cf164514c6d94435a4a7f669ae"}
Sep 11 11:53:23.589890 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:23.589836    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68d47ddfc4-rxsjk" podStartSLOduration=1.589818283 podStartE2EDuration="1.589818283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:23.589188141 +0000 UTC m=+112.089908410" watchObservedRunningTime="2026-09-11 11:53:23.589818283 +0000 UTC m=+112.090538613"
Sep 11 11:53:24.550572 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:24.550538    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-68b56d7dcb-4slw2"
Sep 11 11:53:27.046880 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:27.046838    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:27.047416 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:27.046899    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:27.051605 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:27.051584    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:27.591461 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:27.591432    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:33.043276 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:33.043242    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:33.043752 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:33.043319    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:33.047987 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:33.047966    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:33.617780 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:33.617748    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:53:33.660877 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:33.660845    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d76cfd779-pttwl"]
Sep 11 11:53:41.873639 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:41.873586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:53:41.875974 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:41.875953    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142371cb-79c5-44b6-9e7d-9c01359b29f4-metrics-certs\") pod \"network-metrics-daemon-w98wm\" (UID: \"142371cb-79c5-44b6-9e7d-9c01359b29f4\") " pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:53:42.135634 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:42.135552    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-74fqm\""
Sep 11 11:53:42.143885 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:42.143860    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w98wm"
Sep 11 11:53:42.418237 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:42.418152    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w98wm"]
Sep 11 11:53:42.421485 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:53:42.421453    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142371cb_79c5_44b6_9e7d_9c01359b29f4.slice/crio-b0bbfe05eed1888628f1fa079662d69bad6dfea7285124b51e0ad4d543a7924a WatchSource:0}: Error finding container b0bbfe05eed1888628f1fa079662d69bad6dfea7285124b51e0ad4d543a7924a: Status 404 returned error can't find the container with id b0bbfe05eed1888628f1fa079662d69bad6dfea7285124b51e0ad4d543a7924a
Sep 11 11:53:42.648726 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:42.648688    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w98wm" event={"ID":"142371cb-79c5-44b6-9e7d-9c01359b29f4","Type":"ContainerStarted","Data":"b0bbfe05eed1888628f1fa079662d69bad6dfea7285124b51e0ad4d543a7924a"}
Sep 11 11:53:43.655054 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:43.654964    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w98wm" event={"ID":"142371cb-79c5-44b6-9e7d-9c01359b29f4","Type":"ContainerStarted","Data":"6200ab6410857251aad2572e5340a3506a13a5ccc85a63f73f1fea8e61dfe3b0"}
Sep 11 11:53:43.655054 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:43.655012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w98wm" event={"ID":"142371cb-79c5-44b6-9e7d-9c01359b29f4","Type":"ContainerStarted","Data":"c1cd1600761fed785bbbc46ec42e475197c550515d3770a5fc90c306747afa44"}
Sep 11 11:53:43.656580 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:43.656555    2562 generic.go:356] "Generic (PLEG): container finished" podID="09200696-2c09-4c98-8b6e-6d3116c2319a" containerID="bb2608d1a1821a7780ab61013f11309c1f957ef660e162cc546448b92ad3c9c8" exitCode=0
Sep 11 11:53:43.656672 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:43.656625    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-znzgz" event={"ID":"09200696-2c09-4c98-8b6e-6d3116c2319a","Type":"ContainerDied","Data":"bb2608d1a1821a7780ab61013f11309c1f957ef660e162cc546448b92ad3c9c8"}
Sep 11 11:53:43.656931 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:43.656912    2562 scope.go:117] "RemoveContainer" containerID="bb2608d1a1821a7780ab61013f11309c1f957ef660e162cc546448b92ad3c9c8"
Sep 11 11:53:43.658295 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:43.658268    2562 generic.go:356] "Generic (PLEG): container finished" podID="a9479947-3863-4fcd-8b58-ded0146677a9" containerID="96216c9f29256ca8d02629125a9d5aeeb3b2b2a82b64f42978d72ca992771dc3" exitCode=0
Sep 11 11:53:43.658393 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:43.658299    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9" event={"ID":"a9479947-3863-4fcd-8b58-ded0146677a9","Type":"ContainerDied","Data":"96216c9f29256ca8d02629125a9d5aeeb3b2b2a82b64f42978d72ca992771dc3"}
Sep 11 11:53:43.658577 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:43.658564    2562 scope.go:117] "RemoveContainer" containerID="96216c9f29256ca8d02629125a9d5aeeb3b2b2a82b64f42978d72ca992771dc3"
Sep 11 11:53:43.672690 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:43.672508    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w98wm" podStartSLOduration=130.845175744 podStartE2EDuration="2m11.672487778s" totalImagesPullingTime="827.312034ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:43.670455381 +0000 UTC m=+132.171175633" watchObservedRunningTime="2026-09-11 11:53:43.672487778 +0000 UTC m=+132.173208047"
Sep 11 11:53:44.664662 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:44.664622    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wqtf9" event={"ID":"a9479947-3863-4fcd-8b58-ded0146677a9","Type":"ContainerStarted","Data":"8e3d576cde4419a2da49414b882b1e3449bce08fb3820f13a595271977fc0375"}
Sep 11 11:53:44.666524 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:44.666496    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-znzgz" event={"ID":"09200696-2c09-4c98-8b6e-6d3116c2319a","Type":"ContainerStarted","Data":"e71b6a9fa6d39dbc615b9777807a475e701368b9f5431bddfbaf269e1b4c9bcb"}
Sep 11 11:53:58.680011 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:58.679968    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d76cfd779-pttwl" podUID="d2238619-09b8-4913-96d2-7354d40179c6" containerName="console" containerID="cri-o://50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582" gracePeriod=15
Sep 11 11:53:58.924378 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:58.924351    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:59.033115 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033024    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-console-config\") pod \"d2238619-09b8-4913-96d2-7354d40179c6\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") "
Sep 11 11:53:59.033115 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033093    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-oauth-serving-cert\") pod \"d2238619-09b8-4913-96d2-7354d40179c6\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") "
Sep 11 11:53:59.033336 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033135    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-service-ca\") pod \"d2238619-09b8-4913-96d2-7354d40179c6\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") "
Sep 11 11:53:59.033336 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033161    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-oauth-config\") pod \"d2238619-09b8-4913-96d2-7354d40179c6\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") "
Sep 11 11:53:59.033336 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033191    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6hhz\" (UniqueName: \"kubernetes.io/projected/d2238619-09b8-4913-96d2-7354d40179c6-kube-api-access-m6hhz\") pod \"d2238619-09b8-4913-96d2-7354d40179c6\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") "
Sep 11 11:53:59.033336 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033241    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-serving-cert\") pod \"d2238619-09b8-4913-96d2-7354d40179c6\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") "
Sep 11 11:53:59.033336 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033276    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-trusted-ca-bundle\") pod \"d2238619-09b8-4913-96d2-7354d40179c6\" (UID: \"d2238619-09b8-4913-96d2-7354d40179c6\") "
Sep 11 11:53:59.033569 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033532    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-console-config" (OuterVolumeSpecName: "console-config") pod "d2238619-09b8-4913-96d2-7354d40179c6" (UID: "d2238619-09b8-4913-96d2-7354d40179c6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:59.033628 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033601    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-console-config\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:53:59.033683 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033630    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-service-ca" (OuterVolumeSpecName: "service-ca") pod "d2238619-09b8-4913-96d2-7354d40179c6" (UID: "d2238619-09b8-4913-96d2-7354d40179c6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:59.033683 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033647    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d2238619-09b8-4913-96d2-7354d40179c6" (UID: "d2238619-09b8-4913-96d2-7354d40179c6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:59.033925 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.033888    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d2238619-09b8-4913-96d2-7354d40179c6" (UID: "d2238619-09b8-4913-96d2-7354d40179c6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:53:59.035639 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.035610    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d2238619-09b8-4913-96d2-7354d40179c6" (UID: "d2238619-09b8-4913-96d2-7354d40179c6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:53:59.035639 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.035623    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d2238619-09b8-4913-96d2-7354d40179c6" (UID: "d2238619-09b8-4913-96d2-7354d40179c6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:53:59.035762 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.035704    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2238619-09b8-4913-96d2-7354d40179c6-kube-api-access-m6hhz" (OuterVolumeSpecName: "kube-api-access-m6hhz") pod "d2238619-09b8-4913-96d2-7354d40179c6" (UID: "d2238619-09b8-4913-96d2-7354d40179c6"). InnerVolumeSpecName "kube-api-access-m6hhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:53:59.134533 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.134493    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:53:59.134533 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.134526    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-oauth-config\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:53:59.134533 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.134536    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6hhz\" (UniqueName: \"kubernetes.io/projected/d2238619-09b8-4913-96d2-7354d40179c6-kube-api-access-m6hhz\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:53:59.134752 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.134545    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2238619-09b8-4913-96d2-7354d40179c6-console-serving-cert\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:53:59.134752 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.134554    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-trusted-ca-bundle\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:53:59.134752 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.134563    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2238619-09b8-4913-96d2-7354d40179c6-oauth-serving-cert\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:53:59.732896 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.732858    2562 generic.go:356] "Generic (PLEG): container finished" podID="d2238619-09b8-4913-96d2-7354d40179c6" containerID="50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582" exitCode=0
Sep 11 11:53:59.733291 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.732908    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d76cfd779-pttwl"
Sep 11 11:53:59.733291 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.732952    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d76cfd779-pttwl" event={"ID":"d2238619-09b8-4913-96d2-7354d40179c6","Type":"ContainerDied","Data":"50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582"}
Sep 11 11:53:59.733291 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.733010    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d76cfd779-pttwl" event={"ID":"d2238619-09b8-4913-96d2-7354d40179c6","Type":"ContainerDied","Data":"a1344cbde5cb7e57519114ad2c78e26ed5853cbe621292b49dccdd55a756423b"}
Sep 11 11:53:59.733291 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.733037    2562 scope.go:117] "RemoveContainer" containerID="50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582"
Sep 11 11:53:59.741667 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.741645    2562 scope.go:117] "RemoveContainer" containerID="50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582"
Sep 11 11:53:59.741951 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:53:59.741930    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582\": container with ID starting with 50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582 not found: ID does not exist" containerID="50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582"
Sep 11 11:53:59.742020 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.741963    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582"} err="failed to get container status \"50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582\": rpc error: code = NotFound desc = could not find container \"50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582\": container with ID starting with 50333203012d013afdbfbeca95692061a1d65e1015f9b2ab7c5b6a2506ed6582 not found: ID does not exist"
Sep 11 11:53:59.751738 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.751714    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d76cfd779-pttwl"]
Sep 11 11:53:59.755907 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:53:59.755885    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d76cfd779-pttwl"]
Sep 11 11:54:00.027772 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:00.027690    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2238619-09b8-4913-96d2-7354d40179c6" path="/var/lib/kubelet/pods/d2238619-09b8-4913-96d2-7354d40179c6/volumes"
Sep 11 11:54:33.740519 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.740482    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:54:33.740921 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.740792    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2238619-09b8-4913-96d2-7354d40179c6" containerName="console"
Sep 11 11:54:33.740921 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.740803    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2238619-09b8-4913-96d2-7354d40179c6" containerName="console"
Sep 11 11:54:33.740991 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.740923    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2238619-09b8-4913-96d2-7354d40179c6" containerName="console"
Sep 11 11:54:33.745797 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.745769    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.748084 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.748027    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 11:54:33.748084 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.748032    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 11:54:33.748267 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.748090    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 11:54:33.748267 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.748048    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 11:54:33.748368 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.748297    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 11:54:33.748368 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.748328    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 11:54:33.748368 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.748332    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 11:54:33.748527 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.748297    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-n4lvm\""
Sep 11 11:54:33.748663 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.748611    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 11:54:33.752879 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.752550    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 11:54:33.756101 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.756078    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:54:33.830512 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-web-config\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830512 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830519    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830724 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830539    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cc4340a-85dd-491b-8ac1-b080841d6ca0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830724 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cc4340a-85dd-491b-8ac1-b080841d6ca0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830724 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830724 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830700    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cc4340a-85dd-491b-8ac1-b080841d6ca0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830846 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830736    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830846 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830759    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rvfd\" (UniqueName: \"kubernetes.io/projected/8cc4340a-85dd-491b-8ac1-b080841d6ca0-kube-api-access-2rvfd\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830846 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830815    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830935 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830846    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830935 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830888    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cc4340a-85dd-491b-8ac1-b080841d6ca0-config-out\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.830935 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830915    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cc4340a-85dd-491b-8ac1-b080841d6ca0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.831026 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.830936    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.931626 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.931583    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.931626 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.931633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-web-config\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.931848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.931751    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.931848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.931788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cc4340a-85dd-491b-8ac1-b080841d6ca0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.931848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.931807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cc4340a-85dd-491b-8ac1-b080841d6ca0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.931848 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.931838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.932046 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.931888    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cc4340a-85dd-491b-8ac1-b080841d6ca0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.932046 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.931925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.932046 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.931954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rvfd\" (UniqueName: \"kubernetes.io/projected/8cc4340a-85dd-491b-8ac1-b080841d6ca0-kube-api-access-2rvfd\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.932046 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.931988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.932046 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.932032    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.932287 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.932062    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cc4340a-85dd-491b-8ac1-b080841d6ca0-config-out\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.932287 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.932108    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cc4340a-85dd-491b-8ac1-b080841d6ca0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.932287 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.932197    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cc4340a-85dd-491b-8ac1-b080841d6ca0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.932922 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.932892    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cc4340a-85dd-491b-8ac1-b080841d6ca0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.933233 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.933203    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cc4340a-85dd-491b-8ac1-b080841d6ca0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.935077 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.934796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-web-config\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.935077 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.934863    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.935077 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.934974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.935077 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.934987    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cc4340a-85dd-491b-8ac1-b080841d6ca0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.935359 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.935321    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.935815 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.935794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.935815 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.935806    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.935913 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.935884    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cc4340a-85dd-491b-8ac1-b080841d6ca0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.936587 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.936566    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cc4340a-85dd-491b-8ac1-b080841d6ca0-config-out\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:33.939853 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:33.939832    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rvfd\" (UniqueName: \"kubernetes.io/projected/8cc4340a-85dd-491b-8ac1-b080841d6ca0-kube-api-access-2rvfd\") pod \"alertmanager-main-0\" (UID: \"8cc4340a-85dd-491b-8ac1-b080841d6ca0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:34.055244 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:34.055159    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:54:34.336153 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:34.335980    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:54:34.338811 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:54:34.338782    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc4340a_85dd_491b_8ac1_b080841d6ca0.slice/crio-aef66e7dc384497d78df4370d85ecb306517afb49852ba4529f98e3de276160c WatchSource:0}: Error finding container aef66e7dc384497d78df4370d85ecb306517afb49852ba4529f98e3de276160c: Status 404 returned error can't find the container with id aef66e7dc384497d78df4370d85ecb306517afb49852ba4529f98e3de276160c
Sep 11 11:54:34.872265 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:34.872231    2562 generic.go:356] "Generic (PLEG): container finished" podID="8cc4340a-85dd-491b-8ac1-b080841d6ca0" containerID="5d256a685f61afaa6b0071d8e21a0bfcc94a0302a6c1581e6a5c6cffaebace31" exitCode=0
Sep 11 11:54:34.872631 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:34.872331    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc4340a-85dd-491b-8ac1-b080841d6ca0","Type":"ContainerDied","Data":"5d256a685f61afaa6b0071d8e21a0bfcc94a0302a6c1581e6a5c6cffaebace31"}
Sep 11 11:54:34.872631 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:34.872445    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc4340a-85dd-491b-8ac1-b080841d6ca0","Type":"ContainerStarted","Data":"aef66e7dc384497d78df4370d85ecb306517afb49852ba4529f98e3de276160c"}
Sep 11 11:54:36.885884 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:36.885849    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc4340a-85dd-491b-8ac1-b080841d6ca0","Type":"ContainerStarted","Data":"1cdf9a5b8cc181da13d3c0a80fd85713f5cbfc20705bc67f9cf7668ccce63511"}
Sep 11 11:54:36.886277 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:36.885913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc4340a-85dd-491b-8ac1-b080841d6ca0","Type":"ContainerStarted","Data":"17daa1612688dcfe4f66038b8f098450003fa4021d34db954c686765f803702c"}
Sep 11 11:54:36.886277 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:36.885941    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc4340a-85dd-491b-8ac1-b080841d6ca0","Type":"ContainerStarted","Data":"834632e923fa8846ad1b8e57e56e5edfbb79e949ed92c4da217acdeb6ba6030e"}
Sep 11 11:54:36.886277 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:36.885965    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc4340a-85dd-491b-8ac1-b080841d6ca0","Type":"ContainerStarted","Data":"3e9fdabaf12f32a3cb211bb3222de3ba8987e50b4772b79673e18a4ef9d31b02"}
Sep 11 11:54:36.886277 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:36.885988    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc4340a-85dd-491b-8ac1-b080841d6ca0","Type":"ContainerStarted","Data":"9d055eab1f8424696091779f5fa15e3dac6a4ab0b9070d53efb66a203ce47f93"}
Sep 11 11:54:36.886277 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:36.886012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cc4340a-85dd-491b-8ac1-b080841d6ca0","Type":"ContainerStarted","Data":"1e60e9e1ba60af4dcb765a46a80fc3707505aee65f7a2c2d2298b8a2e91efef5"}
Sep 11 11:54:36.909889 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:36.909798    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.361277818 podStartE2EDuration="3.909783049s" totalImagesPullingTime="1.533882938s" totalInitContainerRuntime="14.622293ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:54:36.907778442 +0000 UTC m=+185.408498710" watchObservedRunningTime="2026-09-11 11:54:36.909783049 +0000 UTC m=+185.410503314"
Sep 11 11:54:52.982247 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:52.982210    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d6c9b9d55-9nqlz"]
Sep 11 11:54:52.988683 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:52.988660    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:52.996923 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:52.996895    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d6c9b9d55-9nqlz"]
Sep 11 11:54:53.000498 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.000468    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-oauth-config\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.000616 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.000514    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-oauth-serving-cert\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.000616 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.000562    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-trusted-ca-bundle\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.000616 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.000578    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-serving-cert\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.000616 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.000593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmnp7\" (UniqueName: \"kubernetes.io/projected/da29293c-faca-4fdf-a288-993d69c07cb5-kube-api-access-pmnp7\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.000804 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.000707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-service-ca\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.000804 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.000753    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-console-config\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.101719 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.101677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-oauth-serving-cert\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.101719 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.101729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-trusted-ca-bundle\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.101973 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.101745    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-serving-cert\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.101973 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.101761    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmnp7\" (UniqueName: \"kubernetes.io/projected/da29293c-faca-4fdf-a288-993d69c07cb5-kube-api-access-pmnp7\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.101973 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.101823    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-service-ca\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.102133 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.102099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-console-config\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.102233 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.102199    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-oauth-config\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.102764 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.102618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-oauth-serving-cert\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.102864 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.102812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-service-ca\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.102864 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.102813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-trusted-ca-bundle\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.102982 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.102953    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-console-config\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.104657 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.104634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-serving-cert\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.104747 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.104666    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-oauth-config\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.108963 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.108945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmnp7\" (UniqueName: \"kubernetes.io/projected/da29293c-faca-4fdf-a288-993d69c07cb5-kube-api-access-pmnp7\") pod \"console-7d6c9b9d55-9nqlz\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") " pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.300265 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.300164    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:54:53.570530 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.570502    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d6c9b9d55-9nqlz"]
Sep 11 11:54:53.572870 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:54:53.572835    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda29293c_faca_4fdf_a288_993d69c07cb5.slice/crio-5edc7a275ae1786217107777d6b4e5d48da14f0477a159299f82e5edb067b475 WatchSource:0}: Error finding container 5edc7a275ae1786217107777d6b4e5d48da14f0477a159299f82e5edb067b475: Status 404 returned error can't find the container with id 5edc7a275ae1786217107777d6b4e5d48da14f0477a159299f82e5edb067b475
Sep 11 11:54:53.963027 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.962983    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6c9b9d55-9nqlz" event={"ID":"da29293c-faca-4fdf-a288-993d69c07cb5","Type":"ContainerStarted","Data":"26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550"}
Sep 11 11:54:53.963219 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.963056    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6c9b9d55-9nqlz" event={"ID":"da29293c-faca-4fdf-a288-993d69c07cb5","Type":"ContainerStarted","Data":"5edc7a275ae1786217107777d6b4e5d48da14f0477a159299f82e5edb067b475"}
Sep 11 11:54:53.979028 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:54:53.978979    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d6c9b9d55-9nqlz" podStartSLOduration=1.978962558 podStartE2EDuration="1.978962558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:54:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:54:53.977469856 +0000 UTC m=+202.478190124" watchObservedRunningTime="2026-09-11 11:54:53.978962558 +0000 UTC m=+202.479682826"
Sep 11 11:55:03.300399 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:03.300355    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:55:03.300881 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:03.300428    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:55:03.305418 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:03.305380    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:55:04.011159 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:04.011131    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:55:04.055480 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:04.055448    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68d47ddfc4-rxsjk"]
Sep 11 11:55:29.074406 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.074359    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68d47ddfc4-rxsjk" podUID="18e11096-18b2-46b4-ad33-b2de2911bfc4" containerName="console" containerID="cri-o://7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775" gracePeriod=15
Sep 11 11:55:29.320637 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.320613    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:55:29.419468 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.419380    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-trusted-ca-bundle\") pod \"18e11096-18b2-46b4-ad33-b2de2911bfc4\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") "
Sep 11 11:55:29.419468 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.419427    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wltnq\" (UniqueName: \"kubernetes.io/projected/18e11096-18b2-46b4-ad33-b2de2911bfc4-kube-api-access-wltnq\") pod \"18e11096-18b2-46b4-ad33-b2de2911bfc4\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") "
Sep 11 11:55:29.419468 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.419447    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-service-ca\") pod \"18e11096-18b2-46b4-ad33-b2de2911bfc4\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") "
Sep 11 11:55:29.419468 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.419470    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-oauth-config\") pod \"18e11096-18b2-46b4-ad33-b2de2911bfc4\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") "
Sep 11 11:55:29.419752 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.419518    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-serving-cert\") pod \"18e11096-18b2-46b4-ad33-b2de2911bfc4\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") "
Sep 11 11:55:29.419752 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.419549    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-config\") pod \"18e11096-18b2-46b4-ad33-b2de2911bfc4\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") "
Sep 11 11:55:29.419752 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.419639    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-oauth-serving-cert\") pod \"18e11096-18b2-46b4-ad33-b2de2911bfc4\" (UID: \"18e11096-18b2-46b4-ad33-b2de2911bfc4\") "
Sep 11 11:55:29.419900 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.419870    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "18e11096-18b2-46b4-ad33-b2de2911bfc4" (UID: "18e11096-18b2-46b4-ad33-b2de2911bfc4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:55:29.419900 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.419887    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-service-ca" (OuterVolumeSpecName: "service-ca") pod "18e11096-18b2-46b4-ad33-b2de2911bfc4" (UID: "18e11096-18b2-46b4-ad33-b2de2911bfc4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:55:29.420172 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.420128    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "18e11096-18b2-46b4-ad33-b2de2911bfc4" (UID: "18e11096-18b2-46b4-ad33-b2de2911bfc4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:55:29.420337 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.420184    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-config" (OuterVolumeSpecName: "console-config") pod "18e11096-18b2-46b4-ad33-b2de2911bfc4" (UID: "18e11096-18b2-46b4-ad33-b2de2911bfc4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:55:29.421732 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.421709    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "18e11096-18b2-46b4-ad33-b2de2911bfc4" (UID: "18e11096-18b2-46b4-ad33-b2de2911bfc4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:55:29.422123 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.422099    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "18e11096-18b2-46b4-ad33-b2de2911bfc4" (UID: "18e11096-18b2-46b4-ad33-b2de2911bfc4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:55:29.422198 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.422137    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e11096-18b2-46b4-ad33-b2de2911bfc4-kube-api-access-wltnq" (OuterVolumeSpecName: "kube-api-access-wltnq") pod "18e11096-18b2-46b4-ad33-b2de2911bfc4" (UID: "18e11096-18b2-46b4-ad33-b2de2911bfc4"). InnerVolumeSpecName "kube-api-access-wltnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:55:29.520403 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.520355    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-oauth-serving-cert\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:55:29.520403 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.520395    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-trusted-ca-bundle\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:55:29.520403 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.520405    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wltnq\" (UniqueName: \"kubernetes.io/projected/18e11096-18b2-46b4-ad33-b2de2911bfc4-kube-api-access-wltnq\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:55:29.520403 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.520414    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:55:29.520663 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.520425    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-oauth-config\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:55:29.520663 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.520434    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-serving-cert\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:55:29.520663 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:29.520442    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18e11096-18b2-46b4-ad33-b2de2911bfc4-console-config\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:55:30.115795 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:30.115757    2562 generic.go:356] "Generic (PLEG): container finished" podID="18e11096-18b2-46b4-ad33-b2de2911bfc4" containerID="7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775" exitCode=0
Sep 11 11:55:30.116173 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:30.115823    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68d47ddfc4-rxsjk"
Sep 11 11:55:30.116173 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:30.115840    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68d47ddfc4-rxsjk" event={"ID":"18e11096-18b2-46b4-ad33-b2de2911bfc4","Type":"ContainerDied","Data":"7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775"}
Sep 11 11:55:30.116173 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:30.115892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68d47ddfc4-rxsjk" event={"ID":"18e11096-18b2-46b4-ad33-b2de2911bfc4","Type":"ContainerDied","Data":"f794bc565a8209317b4d5c4d902576ff3cdb41cf164514c6d94435a4a7f669ae"}
Sep 11 11:55:30.116173 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:30.115916    2562 scope.go:117] "RemoveContainer" containerID="7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775"
Sep 11 11:55:30.124396 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:30.124378    2562 scope.go:117] "RemoveContainer" containerID="7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775"
Sep 11 11:55:30.124669 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:55:30.124647    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775\": container with ID starting with 7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775 not found: ID does not exist" containerID="7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775"
Sep 11 11:55:30.124731 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:30.124676    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775"} err="failed to get container status \"7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775\": rpc error: code = NotFound desc = could not find container \"7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775\": container with ID starting with 7b74a35a037088ed6d09c97cee565772e52dd4e774cf90239c1b8c91003dd775 not found: ID does not exist"
Sep 11 11:55:30.130987 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:30.130966    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68d47ddfc4-rxsjk"]
Sep 11 11:55:30.134742 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:30.134724    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68d47ddfc4-rxsjk"]
Sep 11 11:55:32.023738 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:32.023704    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e11096-18b2-46b4-ad33-b2de2911bfc4" path="/var/lib/kubelet/pods/18e11096-18b2-46b4-ad33-b2de2911bfc4/volumes"
Sep 11 11:55:44.230202 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.230164    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hhmkf"]
Sep 11 11:55:44.230847 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.230526    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e11096-18b2-46b4-ad33-b2de2911bfc4" containerName="console"
Sep 11 11:55:44.230847 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.230540    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e11096-18b2-46b4-ad33-b2de2911bfc4" containerName="console"
Sep 11 11:55:44.230847 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.230604    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e11096-18b2-46b4-ad33-b2de2911bfc4" containerName="console"
Sep 11 11:55:44.235068 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.235052    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf"
Sep 11 11:55:44.237141 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.237114    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 11:55:44.237677 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.237661    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bx8cg\""
Sep 11 11:55:44.237744 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.237681    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 11:55:44.243978 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.243954    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hhmkf"]
Sep 11 11:55:44.349266 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.349230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4884ca67-eba6-4b3d-89b3-81c72377394d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hhmkf\" (UID: \"4884ca67-eba6-4b3d-89b3-81c72377394d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf"
Sep 11 11:55:44.349459 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.349272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68mqw\" (UniqueName: \"kubernetes.io/projected/4884ca67-eba6-4b3d-89b3-81c72377394d-kube-api-access-68mqw\") pod \"cert-manager-webhook-6486ffd997-hhmkf\" (UID: \"4884ca67-eba6-4b3d-89b3-81c72377394d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf"
Sep 11 11:55:44.450043 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.449999    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4884ca67-eba6-4b3d-89b3-81c72377394d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hhmkf\" (UID: \"4884ca67-eba6-4b3d-89b3-81c72377394d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf"
Sep 11 11:55:44.450226 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.450052    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68mqw\" (UniqueName: \"kubernetes.io/projected/4884ca67-eba6-4b3d-89b3-81c72377394d-kube-api-access-68mqw\") pod \"cert-manager-webhook-6486ffd997-hhmkf\" (UID: \"4884ca67-eba6-4b3d-89b3-81c72377394d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf"
Sep 11 11:55:44.457589 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.457558    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4884ca67-eba6-4b3d-89b3-81c72377394d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hhmkf\" (UID: \"4884ca67-eba6-4b3d-89b3-81c72377394d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf"
Sep 11 11:55:44.457793 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.457772    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68mqw\" (UniqueName: \"kubernetes.io/projected/4884ca67-eba6-4b3d-89b3-81c72377394d-kube-api-access-68mqw\") pod \"cert-manager-webhook-6486ffd997-hhmkf\" (UID: \"4884ca67-eba6-4b3d-89b3-81c72377394d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf"
Sep 11 11:55:44.544174 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.544081    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf"
Sep 11 11:55:44.813007 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:44.812910    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hhmkf"]
Sep 11 11:55:44.816954 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:55:44.816923    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4884ca67_eba6_4b3d_89b3_81c72377394d.slice/crio-907dc344c1ca469e1d5e8e4cda7e01cb45d1ff9248caad00306814c1c6cdd3de WatchSource:0}: Error finding container 907dc344c1ca469e1d5e8e4cda7e01cb45d1ff9248caad00306814c1c6cdd3de: Status 404 returned error can't find the container with id 907dc344c1ca469e1d5e8e4cda7e01cb45d1ff9248caad00306814c1c6cdd3de
Sep 11 11:55:45.180277 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:45.180238    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf" event={"ID":"4884ca67-eba6-4b3d-89b3-81c72377394d","Type":"ContainerStarted","Data":"907dc344c1ca469e1d5e8e4cda7e01cb45d1ff9248caad00306814c1c6cdd3de"}
Sep 11 11:55:48.196526 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:48.196440    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf" event={"ID":"4884ca67-eba6-4b3d-89b3-81c72377394d","Type":"ContainerStarted","Data":"e72d2492dc1bf1757e9d2f558db7640e6fc6a139500671c3b09ae36819bfcec9"}
Sep 11 11:55:48.196526 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:48.196506    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf"
Sep 11 11:55:48.211354 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:48.211286    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf" podStartSLOduration=1.102410537 podStartE2EDuration="4.211268041s" totalImagesPullingTime="3.108857504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:48.209749685 +0000 UTC m=+256.710469952" watchObservedRunningTime="2026-09-11 11:55:48.211268041 +0000 UTC m=+256.711988308"
Sep 11 11:55:54.203515 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:55:54.203445    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hhmkf"
Sep 11 11:56:17.952722 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:17.952688    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"]
Sep 11 11:56:17.956343 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:17.956319    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:17.958811 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:17.958787    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 11:56:17.958934 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:17.958874    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 11:56:17.958934 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:17.958876    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 11:56:17.959055 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:17.958980    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ns5m5\""
Sep 11 11:56:17.959113 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:17.959081    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 11:56:17.977658 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:17.977636    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"]
Sep 11 11:56:18.035588 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.035559    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.035744 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.035597    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-webhook-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.035744 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.035626    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsrnv\" (UniqueName: \"kubernetes.io/projected/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-kube-api-access-fsrnv\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.035744 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.035686    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-manifests\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.035744 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.035723    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-tmp\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.136163 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.136124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.136380 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.136173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-webhook-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.136380 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.136201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsrnv\" (UniqueName: \"kubernetes.io/projected/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-kube-api-access-fsrnv\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.136380 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.136223    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-manifests\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.136380 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.136242    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-tmp\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.136769 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.136744    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-manifests\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.138684 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.138654    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-tmp\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.138800 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.138714    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-webhook-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.138864 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.138800    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.143044 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.143021    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsrnv\" (UniqueName: \"kubernetes.io/projected/e9f7f12a-f7ee-4702-8dfe-22145fb7939a-kube-api-access-fsrnv\") pod \"opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j\" (UID: \"e9f7f12a-f7ee-4702-8dfe-22145fb7939a\") " pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.266624 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.266535    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:18.542958 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:18.542937    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"]
Sep 11 11:56:18.545470 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:56:18.545439    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f7f12a_f7ee_4702_8dfe_22145fb7939a.slice/crio-21ed49f28b62d8f399ca954422461582ceaafda2beef496b73f7de08334af7cf WatchSource:0}: Error finding container 21ed49f28b62d8f399ca954422461582ceaafda2beef496b73f7de08334af7cf: Status 404 returned error can't find the container with id 21ed49f28b62d8f399ca954422461582ceaafda2beef496b73f7de08334af7cf
Sep 11 11:56:19.334744 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:19.334696    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j" event={"ID":"e9f7f12a-f7ee-4702-8dfe-22145fb7939a","Type":"ContainerStarted","Data":"21ed49f28b62d8f399ca954422461582ceaafda2beef496b73f7de08334af7cf"}
Sep 11 11:56:22.351573 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:22.351537    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j" event={"ID":"e9f7f12a-f7ee-4702-8dfe-22145fb7939a","Type":"ContainerStarted","Data":"f39fe2966442a9d2534b1bd119c1d739081bd0f33b3044a86001e1cb0992cfa7"}
Sep 11 11:56:23.357121 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:23.357035    2562 generic.go:356] "Generic (PLEG): container finished" podID="e9f7f12a-f7ee-4702-8dfe-22145fb7939a" containerID="f39fe2966442a9d2534b1bd119c1d739081bd0f33b3044a86001e1cb0992cfa7" exitCode=0
Sep 11 11:56:23.357555 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:23.357116    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j" event={"ID":"e9f7f12a-f7ee-4702-8dfe-22145fb7939a","Type":"ContainerDied","Data":"f39fe2966442a9d2534b1bd119c1d739081bd0f33b3044a86001e1cb0992cfa7"}
Sep 11 11:56:24.363940 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:24.363901    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j" event={"ID":"e9f7f12a-f7ee-4702-8dfe-22145fb7939a","Type":"ContainerStarted","Data":"36a31ace58fa3ca7c0bb6fda6c888502c134781365423a444f634d8083771cf3"}
Sep 11 11:56:24.364407 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:24.364058    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:24.386232 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:24.386180    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j" podStartSLOduration=2.5744347640000003 podStartE2EDuration="7.386166216s" totalImagesPullingTime="3.752354233s" totalInitContainerRuntime="1.059377219s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:24.385833535 +0000 UTC m=+292.886553804" watchObservedRunningTime="2026-09-11 11:56:24.386166216 +0000 UTC m=+292.886886483"
Sep 11 11:56:31.936550 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:31.936522    2562 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 11:56:35.370783 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:35.370749    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j"
Sep 11 11:56:41.609993 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.609957    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"]
Sep 11 11:56:41.613299 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.613282    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.615881 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.615859    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 11:56:41.616010 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.615864    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:56:41.617291 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.617274    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 11:56:41.617389 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.617361    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7dzh6\""
Sep 11 11:56:41.617450 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.617386    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 11:56:41.617450 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.617403    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 11:56:41.623763 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.623740    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"]
Sep 11 11:56:41.738196 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.738155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/104e6446-c5d4-42d8-b32a-434f2ca4a043-metrics-cert\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.738196 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.738202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svdjp\" (UniqueName: \"kubernetes.io/projected/104e6446-c5d4-42d8-b32a-434f2ca4a043-kube-api-access-svdjp\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.738431 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.738330    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/104e6446-c5d4-42d8-b32a-434f2ca4a043-manager-config\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.738431 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.738394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/104e6446-c5d4-42d8-b32a-434f2ca4a043-cert\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.839169 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.839135    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/104e6446-c5d4-42d8-b32a-434f2ca4a043-manager-config\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.839328 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.839188    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/104e6446-c5d4-42d8-b32a-434f2ca4a043-cert\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.839328 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.839219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/104e6446-c5d4-42d8-b32a-434f2ca4a043-metrics-cert\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.839328 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.839245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svdjp\" (UniqueName: \"kubernetes.io/projected/104e6446-c5d4-42d8-b32a-434f2ca4a043-kube-api-access-svdjp\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.839803 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.839780    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/104e6446-c5d4-42d8-b32a-434f2ca4a043-manager-config\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.841669 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.841635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/104e6446-c5d4-42d8-b32a-434f2ca4a043-metrics-cert\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.841768 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.841735    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/104e6446-c5d4-42d8-b32a-434f2ca4a043-cert\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.847610 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.847590    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svdjp\" (UniqueName: \"kubernetes.io/projected/104e6446-c5d4-42d8-b32a-434f2ca4a043-kube-api-access-svdjp\") pod \"lws-controller-manager-56c7cddbd4-lctbg\" (UID: \"104e6446-c5d4-42d8-b32a-434f2ca4a043\") " pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:41.922886 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:41.922860    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:42.199328 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:42.199249    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"]
Sep 11 11:56:42.202129 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:56:42.202099    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod104e6446_c5d4_42d8_b32a_434f2ca4a043.slice/crio-619f687d32c99c33a63c8cc6482b9a76f4dba2ca78d18779dac32efa2b164f6a WatchSource:0}: Error finding container 619f687d32c99c33a63c8cc6482b9a76f4dba2ca78d18779dac32efa2b164f6a: Status 404 returned error can't find the container with id 619f687d32c99c33a63c8cc6482b9a76f4dba2ca78d18779dac32efa2b164f6a
Sep 11 11:56:42.204045 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:42.204027    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 11:56:42.444590 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:42.444553    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg" event={"ID":"104e6446-c5d4-42d8-b32a-434f2ca4a043","Type":"ContainerStarted","Data":"619f687d32c99c33a63c8cc6482b9a76f4dba2ca78d18779dac32efa2b164f6a"}
Sep 11 11:56:45.461556 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:45.461520    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg" event={"ID":"104e6446-c5d4-42d8-b32a-434f2ca4a043","Type":"ContainerStarted","Data":"609d7b26ac9b46ed55601bd7e6f0cd4769d383513338aedda532cf84a79bd565"}
Sep 11 11:56:45.462000 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:45.461663    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:45.477220 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:45.477173    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg" podStartSLOduration=2.142166127 podStartE2EDuration="4.477158272s" totalImagesPullingTime="2.334992145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:45.475672449 +0000 UTC m=+313.976392716" watchObservedRunningTime="2026-09-11 11:56:45.477158272 +0000 UTC m=+313.977878606"
Sep 11 11:56:55.354260 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.354219    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"]
Sep 11 11:56:55.357675 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.357650    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.360119 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.360093    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 11:56:55.360296 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.360092    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 11:56:55.360296 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.360093    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 11:56:55.360296 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.360091    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 11:56:55.360296 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.360221    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 11:56:55.360600 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.360463    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 11:56:55.360681 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.360659    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jszth\""
Sep 11 11:56:55.375580 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.375546    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"]
Sep 11 11:56:55.458391 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.458350    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.458391 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.458392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.458614 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.458416    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/672739d3-4ea1-4d9a-8184-3c87701a061f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.458614 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.458497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/672739d3-4ea1-4d9a-8184-3c87701a061f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.458614 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.458582    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.458614 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.458604    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zmzk\" (UniqueName: \"kubernetes.io/projected/672739d3-4ea1-4d9a-8184-3c87701a061f-kube-api-access-6zmzk\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.458749 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.458694    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.559359 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.559288    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.559553 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.559367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.559553 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.559391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.559553 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.559412    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/672739d3-4ea1-4d9a-8184-3c87701a061f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.559553 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.559440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/672739d3-4ea1-4d9a-8184-3c87701a061f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.559553 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.559504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.559553 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.559522    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zmzk\" (UniqueName: \"kubernetes.io/projected/672739d3-4ea1-4d9a-8184-3c87701a061f-kube-api-access-6zmzk\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.560236 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.560202    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.561912 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.561881    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/672739d3-4ea1-4d9a-8184-3c87701a061f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.562014 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.561927    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.562252 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.562231    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/672739d3-4ea1-4d9a-8184-3c87701a061f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.562289 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.562245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.568700 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.568673    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/672739d3-4ea1-4d9a-8184-3c87701a061f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.568792 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.568673    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zmzk\" (UniqueName: \"kubernetes.io/projected/672739d3-4ea1-4d9a-8184-3c87701a061f-kube-api-access-6zmzk\") pod \"istiod-openshift-gateway-5677ccb49b-cqzqm\" (UID: \"672739d3-4ea1-4d9a-8184-3c87701a061f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.667569 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.667464    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:55.948032 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:55.947859    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"]
Sep 11 11:56:55.950383 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:56:55.950349    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod672739d3_4ea1_4d9a_8184_3c87701a061f.slice/crio-58eea817683267bf9d0d75806a16e2e6babfcf4427cfb62db2629cf89ea66163 WatchSource:0}: Error finding container 58eea817683267bf9d0d75806a16e2e6babfcf4427cfb62db2629cf89ea66163: Status 404 returned error can't find the container with id 58eea817683267bf9d0d75806a16e2e6babfcf4427cfb62db2629cf89ea66163
Sep 11 11:56:56.467944 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:56.467909    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-56c7cddbd4-lctbg"
Sep 11 11:56:56.514503 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:56.514466    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm" event={"ID":"672739d3-4ea1-4d9a-8184-3c87701a061f","Type":"ContainerStarted","Data":"58eea817683267bf9d0d75806a16e2e6babfcf4427cfb62db2629cf89ea66163"}
Sep 11 11:56:58.639598 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:58.639559    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 11:56:58.639836 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:58.639631    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 11:56:59.533561 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:59.533525    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm" event={"ID":"672739d3-4ea1-4d9a-8184-3c87701a061f","Type":"ContainerStarted","Data":"3a315e65946215051439b5c984415718e4aaa1482b505a0778fa1406fb896665"}
Sep 11 11:56:59.533759 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:59.533682    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:56:59.535319 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:59.535280    2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-cqzqm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 11:56:59.535428 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:59.535337    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm" podUID="672739d3-4ea1-4d9a-8184-3c87701a061f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 11:56:59.553891 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:56:59.553835    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm" podStartSLOduration=1.8669629699999999 podStartE2EDuration="4.553820022s" totalImagesPullingTime="2.686857052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:56:59.551718273 +0000 UTC m=+328.052438541" watchObservedRunningTime="2026-09-11 11:56:59.553820022 +0000 UTC m=+328.054540290"
Sep 11 11:57:00.543899 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:57:00.543869    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cqzqm"
Sep 11 11:58:22.697970 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:22.697934    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d6c9b9d55-9nqlz"]
Sep 11 11:58:34.731720 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.731682    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"]
Sep 11 11:58:34.735057 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.735034    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:34.737291 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.737264    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 11:58:34.737628 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.737459    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 11:58:34.737628 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.737583    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xbl99\""
Sep 11 11:58:34.742596 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.742572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adba3d10-f788-4581-aeae-d5ef195a03e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-8nhjf\" (UID: \"adba3d10-f788-4581-aeae-d5ef195a03e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:34.742699 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.742600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f65jd\" (UniqueName: \"kubernetes.io/projected/adba3d10-f788-4581-aeae-d5ef195a03e9-kube-api-access-f65jd\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-8nhjf\" (UID: \"adba3d10-f788-4581-aeae-d5ef195a03e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:34.750164 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.750138    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"]
Sep 11 11:58:34.843392 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.843360    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adba3d10-f788-4581-aeae-d5ef195a03e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-8nhjf\" (UID: \"adba3d10-f788-4581-aeae-d5ef195a03e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:34.843392 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.843398    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f65jd\" (UniqueName: \"kubernetes.io/projected/adba3d10-f788-4581-aeae-d5ef195a03e9-kube-api-access-f65jd\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-8nhjf\" (UID: \"adba3d10-f788-4581-aeae-d5ef195a03e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:34.843808 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.843787    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adba3d10-f788-4581-aeae-d5ef195a03e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-8nhjf\" (UID: \"adba3d10-f788-4581-aeae-d5ef195a03e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:34.847192 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.847166    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"]
Sep 11 11:58:34.847424 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:58:34.847405    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-f65jd], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf" podUID="adba3d10-f788-4581-aeae-d5ef195a03e9"
Sep 11 11:58:34.852115 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.852094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f65jd\" (UniqueName: \"kubernetes.io/projected/adba3d10-f788-4581-aeae-d5ef195a03e9-kube-api-access-f65jd\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-8nhjf\" (UID: \"adba3d10-f788-4581-aeae-d5ef195a03e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:34.955221 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.955187    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:34.962193 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:34.962172    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:35.045482 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:35.045404    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adba3d10-f788-4581-aeae-d5ef195a03e9-extensions-socket-volume\") pod \"adba3d10-f788-4581-aeae-d5ef195a03e9\" (UID: \"adba3d10-f788-4581-aeae-d5ef195a03e9\") "
Sep 11 11:58:35.045482 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:35.045470    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f65jd\" (UniqueName: \"kubernetes.io/projected/adba3d10-f788-4581-aeae-d5ef195a03e9-kube-api-access-f65jd\") pod \"adba3d10-f788-4581-aeae-d5ef195a03e9\" (UID: \"adba3d10-f788-4581-aeae-d5ef195a03e9\") "
Sep 11 11:58:35.046141 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:35.045683    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adba3d10-f788-4581-aeae-d5ef195a03e9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "adba3d10-f788-4581-aeae-d5ef195a03e9" (UID: "adba3d10-f788-4581-aeae-d5ef195a03e9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:58:35.047624 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:35.047595    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adba3d10-f788-4581-aeae-d5ef195a03e9-kube-api-access-f65jd" (OuterVolumeSpecName: "kube-api-access-f65jd") pod "adba3d10-f788-4581-aeae-d5ef195a03e9" (UID: "adba3d10-f788-4581-aeae-d5ef195a03e9"). InnerVolumeSpecName "kube-api-access-f65jd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:58:35.147333 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:35.147269    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f65jd\" (UniqueName: \"kubernetes.io/projected/adba3d10-f788-4581-aeae-d5ef195a03e9-kube-api-access-f65jd\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:58:35.147333 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:35.147299    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adba3d10-f788-4581-aeae-d5ef195a03e9-extensions-socket-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:58:35.685461 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:35.685424    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"]
Sep 11 11:58:35.703773 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:35.703742    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"]
Sep 11 11:58:35.960293 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:35.960199    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:35.962023 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:58:35.961990    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-8nhjf\" is forbidden: User \"system:node:ip-10-0-136-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-222.ec2.internal' and this object" podUID="adba3d10-f788-4581-aeae-d5ef195a03e9" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:35.967677 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:58:35.967653    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-8nhjf\" is forbidden: User \"system:node:ip-10-0-136-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-222.ec2.internal' and this object" podUID="adba3d10-f788-4581-aeae-d5ef195a03e9" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-8nhjf"
Sep 11 11:58:36.024749 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:36.024716    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adba3d10-f788-4581-aeae-d5ef195a03e9" path="/var/lib/kubelet/pods/adba3d10-f788-4581-aeae-d5ef195a03e9/volumes"
Sep 11 11:58:47.717346 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:47.717285    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d6c9b9d55-9nqlz" podUID="da29293c-faca-4fdf-a288-993d69c07cb5" containerName="console" containerID="cri-o://26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550" gracePeriod=15
Sep 11 11:58:47.954048 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:47.954022    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:58:48.017871 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.017779    2562 generic.go:356] "Generic (PLEG): container finished" podID="da29293c-faca-4fdf-a288-993d69c07cb5" containerID="26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550" exitCode=0
Sep 11 11:58:48.017871 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.017821    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6c9b9d55-9nqlz"
Sep 11 11:58:48.017871 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.017850    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6c9b9d55-9nqlz" event={"ID":"da29293c-faca-4fdf-a288-993d69c07cb5","Type":"ContainerDied","Data":"26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550"}
Sep 11 11:58:48.018110 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.017892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6c9b9d55-9nqlz" event={"ID":"da29293c-faca-4fdf-a288-993d69c07cb5","Type":"ContainerDied","Data":"5edc7a275ae1786217107777d6b4e5d48da14f0477a159299f82e5edb067b475"}
Sep 11 11:58:48.018110 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.017913    2562 scope.go:117] "RemoveContainer" containerID="26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550"
Sep 11 11:58:48.027488 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.027468    2562 scope.go:117] "RemoveContainer" containerID="26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550"
Sep 11 11:58:48.027769 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:58:48.027750    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550\": container with ID starting with 26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550 not found: ID does not exist" containerID="26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550"
Sep 11 11:58:48.027818 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.027782    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550"} err="failed to get container status \"26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550\": rpc error: code = NotFound desc = could not find container \"26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550\": container with ID starting with 26e083148cdf67191fec14a25d469a52cf31c8005bc96d290612a585571d5550 not found: ID does not exist"
Sep 11 11:58:48.058086 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058056    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-oauth-serving-cert\") pod \"da29293c-faca-4fdf-a288-993d69c07cb5\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") "
Sep 11 11:58:48.058167 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058095    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-oauth-config\") pod \"da29293c-faca-4fdf-a288-993d69c07cb5\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") "
Sep 11 11:58:48.058167 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058133    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-trusted-ca-bundle\") pod \"da29293c-faca-4fdf-a288-993d69c07cb5\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") "
Sep 11 11:58:48.058237 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058186    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-console-config\") pod \"da29293c-faca-4fdf-a288-993d69c07cb5\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") "
Sep 11 11:58:48.058237 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058226    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-service-ca\") pod \"da29293c-faca-4fdf-a288-993d69c07cb5\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") "
Sep 11 11:58:48.058336 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058280    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmnp7\" (UniqueName: \"kubernetes.io/projected/da29293c-faca-4fdf-a288-993d69c07cb5-kube-api-access-pmnp7\") pod \"da29293c-faca-4fdf-a288-993d69c07cb5\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") "
Sep 11 11:58:48.058395 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058349    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-serving-cert\") pod \"da29293c-faca-4fdf-a288-993d69c07cb5\" (UID: \"da29293c-faca-4fdf-a288-993d69c07cb5\") "
Sep 11 11:58:48.058969 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058612    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "da29293c-faca-4fdf-a288-993d69c07cb5" (UID: "da29293c-faca-4fdf-a288-993d69c07cb5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:58:48.058969 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058666    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-console-config" (OuterVolumeSpecName: "console-config") pod "da29293c-faca-4fdf-a288-993d69c07cb5" (UID: "da29293c-faca-4fdf-a288-993d69c07cb5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:58:48.058969 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058674    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-service-ca" (OuterVolumeSpecName: "service-ca") pod "da29293c-faca-4fdf-a288-993d69c07cb5" (UID: "da29293c-faca-4fdf-a288-993d69c07cb5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:58:48.058969 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058622    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "da29293c-faca-4fdf-a288-993d69c07cb5" (UID: "da29293c-faca-4fdf-a288-993d69c07cb5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:58:48.058969 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.058895    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-oauth-serving-cert\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:58:48.059341 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.059067    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-trusted-ca-bundle\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:58:48.059341 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.059234    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-console-config\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:58:48.059341 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.059253    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da29293c-faca-4fdf-a288-993d69c07cb5-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:58:48.060504 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.060477    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "da29293c-faca-4fdf-a288-993d69c07cb5" (UID: "da29293c-faca-4fdf-a288-993d69c07cb5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:58:48.060801 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.060783    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "da29293c-faca-4fdf-a288-993d69c07cb5" (UID: "da29293c-faca-4fdf-a288-993d69c07cb5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:58:48.060854 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.060797    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da29293c-faca-4fdf-a288-993d69c07cb5-kube-api-access-pmnp7" (OuterVolumeSpecName: "kube-api-access-pmnp7") pod "da29293c-faca-4fdf-a288-993d69c07cb5" (UID: "da29293c-faca-4fdf-a288-993d69c07cb5"). InnerVolumeSpecName "kube-api-access-pmnp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:58:48.160474 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.160432    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmnp7\" (UniqueName: \"kubernetes.io/projected/da29293c-faca-4fdf-a288-993d69c07cb5-kube-api-access-pmnp7\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:58:48.160474 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.160465    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-serving-cert\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:58:48.160474 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.160475    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da29293c-faca-4fdf-a288-993d69c07cb5-console-oauth-config\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:58:48.339614 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.339586    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d6c9b9d55-9nqlz"]
Sep 11 11:58:48.343224 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:48.343197    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d6c9b9d55-9nqlz"]
Sep 11 11:58:50.025407 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:58:50.025369    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da29293c-faca-4fdf-a288-993d69c07cb5" path="/var/lib/kubelet/pods/da29293c-faca-4fdf-a288-993d69c07cb5/volumes"
Sep 11 11:59:20.756628 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.756593    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mnd86"]
Sep 11 11:59:20.757093 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.756927    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da29293c-faca-4fdf-a288-993d69c07cb5" containerName="console"
Sep 11 11:59:20.757093 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.756938    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="da29293c-faca-4fdf-a288-993d69c07cb5" containerName="console"
Sep 11 11:59:20.757093 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.756999    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="da29293c-faca-4fdf-a288-993d69c07cb5" containerName="console"
Sep 11 11:59:20.760019 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.760002    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:20.762344 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.762315    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 11:59:20.762869 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.762852    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 11:59:20.762925 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.762853    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kbk8h\""
Sep 11 11:59:20.762925 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.762853    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 11:59:20.766964 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.766740    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mnd86"]
Sep 11 11:59:20.851678 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.851640    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6lpl\" (UniqueName: \"kubernetes.io/projected/34b687f0-66bc-4687-b6fc-1d86deb3d96b-kube-api-access-w6lpl\") pod \"limitador-limitador-8588749977-mnd86\" (UID: \"34b687f0-66bc-4687-b6fc-1d86deb3d96b\") " pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:20.851858 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.851707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34b687f0-66bc-4687-b6fc-1d86deb3d96b-config-file\") pod \"limitador-limitador-8588749977-mnd86\" (UID: \"34b687f0-66bc-4687-b6fc-1d86deb3d96b\") " pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:20.855239 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.855214    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mnd86"]
Sep 11 11:59:20.952725 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.952691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6lpl\" (UniqueName: \"kubernetes.io/projected/34b687f0-66bc-4687-b6fc-1d86deb3d96b-kube-api-access-w6lpl\") pod \"limitador-limitador-8588749977-mnd86\" (UID: \"34b687f0-66bc-4687-b6fc-1d86deb3d96b\") " pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:20.952892 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.952743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34b687f0-66bc-4687-b6fc-1d86deb3d96b-config-file\") pod \"limitador-limitador-8588749977-mnd86\" (UID: \"34b687f0-66bc-4687-b6fc-1d86deb3d96b\") " pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:20.953393 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.953374    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34b687f0-66bc-4687-b6fc-1d86deb3d96b-config-file\") pod \"limitador-limitador-8588749977-mnd86\" (UID: \"34b687f0-66bc-4687-b6fc-1d86deb3d96b\") " pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:20.960245 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:20.960223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6lpl\" (UniqueName: \"kubernetes.io/projected/34b687f0-66bc-4687-b6fc-1d86deb3d96b-kube-api-access-w6lpl\") pod \"limitador-limitador-8588749977-mnd86\" (UID: \"34b687f0-66bc-4687-b6fc-1d86deb3d96b\") " pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:21.070659 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:21.070569    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:21.344952 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:21.344873    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mnd86"]
Sep 11 11:59:21.347928 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:59:21.347899    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b687f0_66bc_4687_b6fc_1d86deb3d96b.slice/crio-a71c616fa4e906e8f6e6f34b1d07effbd49ca2761c439b4dc443f56bdd91fc52 WatchSource:0}: Error finding container a71c616fa4e906e8f6e6f34b1d07effbd49ca2761c439b4dc443f56bdd91fc52: Status 404 returned error can't find the container with id a71c616fa4e906e8f6e6f34b1d07effbd49ca2761c439b4dc443f56bdd91fc52
Sep 11 11:59:22.175683 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:22.175642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-mnd86" event={"ID":"34b687f0-66bc-4687-b6fc-1d86deb3d96b","Type":"ContainerStarted","Data":"a71c616fa4e906e8f6e6f34b1d07effbd49ca2761c439b4dc443f56bdd91fc52"}
Sep 11 11:59:24.188048 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:24.188012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-mnd86" event={"ID":"34b687f0-66bc-4687-b6fc-1d86deb3d96b","Type":"ContainerStarted","Data":"fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4"}
Sep 11 11:59:24.188526 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:24.188124    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:24.202820 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:24.202768    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-mnd86" podStartSLOduration=1.861107381 podStartE2EDuration="4.202749997s" totalImagesPullingTime="2.341642616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:59:24.201633161 +0000 UTC m=+472.702353431" watchObservedRunningTime="2026-09-11 11:59:24.202749997 +0000 UTC m=+472.703470265"
Sep 11 11:59:34.018178 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:34.018125    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mnd86"]
Sep 11 11:59:34.018685 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:34.018450    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-mnd86" podUID="34b687f0-66bc-4687-b6fc-1d86deb3d96b" containerName="limitador" containerID="cri-o://fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4" gracePeriod=30
Sep 11 11:59:34.024294 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:34.024271    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:34.571603 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:34.571578    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:34.675942 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:34.675904    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6lpl\" (UniqueName: \"kubernetes.io/projected/34b687f0-66bc-4687-b6fc-1d86deb3d96b-kube-api-access-w6lpl\") pod \"34b687f0-66bc-4687-b6fc-1d86deb3d96b\" (UID: \"34b687f0-66bc-4687-b6fc-1d86deb3d96b\") "
Sep 11 11:59:34.676102 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:34.675949    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34b687f0-66bc-4687-b6fc-1d86deb3d96b-config-file\") pod \"34b687f0-66bc-4687-b6fc-1d86deb3d96b\" (UID: \"34b687f0-66bc-4687-b6fc-1d86deb3d96b\") "
Sep 11 11:59:34.676375 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:34.676351    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34b687f0-66bc-4687-b6fc-1d86deb3d96b-config-file" (OuterVolumeSpecName: "config-file") pod "34b687f0-66bc-4687-b6fc-1d86deb3d96b" (UID: "34b687f0-66bc-4687-b6fc-1d86deb3d96b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:59:34.678148 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:34.678125    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34b687f0-66bc-4687-b6fc-1d86deb3d96b-kube-api-access-w6lpl" (OuterVolumeSpecName: "kube-api-access-w6lpl") pod "34b687f0-66bc-4687-b6fc-1d86deb3d96b" (UID: "34b687f0-66bc-4687-b6fc-1d86deb3d96b"). InnerVolumeSpecName "kube-api-access-w6lpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:59:34.776779 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:34.776738    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6lpl\" (UniqueName: \"kubernetes.io/projected/34b687f0-66bc-4687-b6fc-1d86deb3d96b-kube-api-access-w6lpl\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:59:34.776779 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:34.776774    2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34b687f0-66bc-4687-b6fc-1d86deb3d96b-config-file\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 11:59:35.238505 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:35.238477    2562 generic.go:356] "Generic (PLEG): container finished" podID="34b687f0-66bc-4687-b6fc-1d86deb3d96b" containerID="fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4" exitCode=0
Sep 11 11:59:35.238904 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:35.238528    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-mnd86"
Sep 11 11:59:35.238904 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:35.238554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-mnd86" event={"ID":"34b687f0-66bc-4687-b6fc-1d86deb3d96b","Type":"ContainerDied","Data":"fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4"}
Sep 11 11:59:35.238904 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:35.238594    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-mnd86" event={"ID":"34b687f0-66bc-4687-b6fc-1d86deb3d96b","Type":"ContainerDied","Data":"a71c616fa4e906e8f6e6f34b1d07effbd49ca2761c439b4dc443f56bdd91fc52"}
Sep 11 11:59:35.238904 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:35.238615    2562 scope.go:117] "RemoveContainer" containerID="fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4"
Sep 11 11:59:35.247350 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:35.247332    2562 scope.go:117] "RemoveContainer" containerID="fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4"
Sep 11 11:59:35.247624 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:59:35.247606    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4\": container with ID starting with fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4 not found: ID does not exist" containerID="fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4"
Sep 11 11:59:35.247674 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:35.247631    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4"} err="failed to get container status \"fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4\": rpc error: code = NotFound desc = could not find container \"fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4\": container with ID starting with fcddac7407823a0b083c4701eb2278372ae1f4a44694a5bf1d0b20f00c9c42a4 not found: ID does not exist"
Sep 11 11:59:35.258943 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:35.258917    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mnd86"]
Sep 11 11:59:35.264979 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:35.264957    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mnd86"]
Sep 11 11:59:36.024064 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.024029    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34b687f0-66bc-4687-b6fc-1d86deb3d96b" path="/var/lib/kubelet/pods/34b687f0-66bc-4687-b6fc-1d86deb3d96b/volumes"
Sep 11 11:59:36.725463 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.725426    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"]
Sep 11 11:59:36.726026 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.726005    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34b687f0-66bc-4687-b6fc-1d86deb3d96b" containerName="limitador"
Sep 11 11:59:36.726071 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.726031    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34b687f0-66bc-4687-b6fc-1d86deb3d96b" containerName="limitador"
Sep 11 11:59:36.726160 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.726146    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34b687f0-66bc-4687-b6fc-1d86deb3d96b" containerName="limitador"
Sep 11 11:59:36.730402 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.730380    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"
Sep 11 11:59:36.732729 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.732693    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 11:59:36.732850 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.732763    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 11:59:36.732850 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.732768    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-l75vs\""
Sep 11 11:59:36.733119 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.733102    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 11:59:36.736936 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.736912    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"]
Sep 11 11:59:36.791674 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.791629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3673b99-ed29-4c4b-b895-2ffaa23d43ca-data\") pod \"postgres-69fb4594cf-fghx7\" (UID: \"f3673b99-ed29-4c4b-b895-2ffaa23d43ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"
Sep 11 11:59:36.791674 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.791672    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpnjf\" (UniqueName: \"kubernetes.io/projected/f3673b99-ed29-4c4b-b895-2ffaa23d43ca-kube-api-access-cpnjf\") pod \"postgres-69fb4594cf-fghx7\" (UID: \"f3673b99-ed29-4c4b-b895-2ffaa23d43ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"
Sep 11 11:59:36.892573 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.892460    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3673b99-ed29-4c4b-b895-2ffaa23d43ca-data\") pod \"postgres-69fb4594cf-fghx7\" (UID: \"f3673b99-ed29-4c4b-b895-2ffaa23d43ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"
Sep 11 11:59:36.892757 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.892606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpnjf\" (UniqueName: \"kubernetes.io/projected/f3673b99-ed29-4c4b-b895-2ffaa23d43ca-kube-api-access-cpnjf\") pod \"postgres-69fb4594cf-fghx7\" (UID: \"f3673b99-ed29-4c4b-b895-2ffaa23d43ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"
Sep 11 11:59:36.892799 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.892775    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3673b99-ed29-4c4b-b895-2ffaa23d43ca-data\") pod \"postgres-69fb4594cf-fghx7\" (UID: \"f3673b99-ed29-4c4b-b895-2ffaa23d43ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"
Sep 11 11:59:36.922026 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:36.921998    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpnjf\" (UniqueName: \"kubernetes.io/projected/f3673b99-ed29-4c4b-b895-2ffaa23d43ca-kube-api-access-cpnjf\") pod \"postgres-69fb4594cf-fghx7\" (UID: \"f3673b99-ed29-4c4b-b895-2ffaa23d43ca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"
Sep 11 11:59:37.042969 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:37.042877    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"
Sep 11 11:59:37.309925 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:37.309842    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"]
Sep 11 11:59:37.313106 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:59:37.313085    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3673b99_ed29_4c4b_b895_2ffaa23d43ca.slice/crio-8837d538dfd4d471a5ed7a84aef07139baa4e8ad2640a486ab6190b57ee8dd8b WatchSource:0}: Error finding container 8837d538dfd4d471a5ed7a84aef07139baa4e8ad2640a486ab6190b57ee8dd8b: Status 404 returned error can't find the container with id 8837d538dfd4d471a5ed7a84aef07139baa4e8ad2640a486ab6190b57ee8dd8b
Sep 11 11:59:38.259351 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:38.259299    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7" event={"ID":"f3673b99-ed29-4c4b-b895-2ffaa23d43ca","Type":"ContainerStarted","Data":"8837d538dfd4d471a5ed7a84aef07139baa4e8ad2640a486ab6190b57ee8dd8b"}
Sep 11 11:59:42.284474 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:42.284381    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7" event={"ID":"f3673b99-ed29-4c4b-b895-2ffaa23d43ca","Type":"ContainerStarted","Data":"bfe87c71d845c27bd37f6e3ef343ce3d725bc83d4355846ae8ca9c22c39ae0b0"}
Sep 11 11:59:42.284843 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:42.284610    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"
Sep 11 11:59:42.299201 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:42.299153    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7" podStartSLOduration=1.6452681949999999 podStartE2EDuration="6.299141221s" totalImagesPullingTime="4.653873026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:59:42.297933525 +0000 UTC m=+490.798653796" watchObservedRunningTime="2026-09-11 11:59:42.299141221 +0000 UTC m=+490.799861489"
Sep 11 11:59:48.318476 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:48.318446    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fghx7"
Sep 11 11:59:55.768790 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.768749    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-646d6b78cf-p9r7l"]
Sep 11 11:59:55.772189 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.772164    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:55.774491 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.774463    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 11:59:55.774491 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.774486    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 11:59:55.774671 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.774465    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fcqsp\""
Sep 11 11:59:55.774977 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.774963    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 11:59:55.783688 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.783658    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-646d6b78cf-p9r7l"]
Sep 11 11:59:55.864908 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.864875    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sfb2\" (UniqueName: \"kubernetes.io/projected/5bf59d37-6955-492e-a417-2376104532b8-kube-api-access-4sfb2\") pod \"maas-controller-646d6b78cf-p9r7l\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") " pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:55.865080 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.864920    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-metrics-certs\") pod \"maas-controller-646d6b78cf-p9r7l\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") " pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:55.865080 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.864954    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-webhook-cert\") pod \"maas-controller-646d6b78cf-p9r7l\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") " pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:55.952057 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.952020    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7db788bf7f-r5x4s"]
Sep 11 11:59:55.955155 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.955138    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:55.965753 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.965727    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-r5x4s"]
Sep 11 11:59:55.965862 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.965843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-metrics-certs\") pod \"maas-controller-646d6b78cf-p9r7l\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") " pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:55.965904 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.965875    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-webhook-cert\") pod \"maas-controller-646d6b78cf-p9r7l\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") " pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:55.965966 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.965955    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sfb2\" (UniqueName: \"kubernetes.io/projected/5bf59d37-6955-492e-a417-2376104532b8-kube-api-access-4sfb2\") pod \"maas-controller-646d6b78cf-p9r7l\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") " pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:55.966069 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:59:55.966053    2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 11 11:59:55.966129 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:59:55.966117    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-webhook-cert podName:5bf59d37-6955-492e-a417-2376104532b8 nodeName:}" failed. No retries permitted until 2026-09-11 11:59:56.466098018 +0000 UTC m=+504.966818272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-webhook-cert") pod "maas-controller-646d6b78cf-p9r7l" (UID: "5bf59d37-6955-492e-a417-2376104532b8") : secret "maas-controller-webhook-cert" not found
Sep 11 11:59:55.968341 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.968297    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-metrics-certs\") pod \"maas-controller-646d6b78cf-p9r7l\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") " pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:55.977160 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:55.977134    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sfb2\" (UniqueName: \"kubernetes.io/projected/5bf59d37-6955-492e-a417-2376104532b8-kube-api-access-4sfb2\") pod \"maas-controller-646d6b78cf-p9r7l\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") " pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:56.067187 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.067108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcb5z\" (UniqueName: \"kubernetes.io/projected/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-kube-api-access-mcb5z\") pod \"maas-controller-7db788bf7f-r5x4s\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") " pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.067187 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.067158    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-webhook-cert\") pod \"maas-controller-7db788bf7f-r5x4s\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") " pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.067384 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.067275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-metrics-certs\") pod \"maas-controller-7db788bf7f-r5x4s\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") " pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.168778 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.168741    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcb5z\" (UniqueName: \"kubernetes.io/projected/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-kube-api-access-mcb5z\") pod \"maas-controller-7db788bf7f-r5x4s\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") " pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.168872 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.168795    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-webhook-cert\") pod \"maas-controller-7db788bf7f-r5x4s\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") " pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.168872 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.168851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-metrics-certs\") pod \"maas-controller-7db788bf7f-r5x4s\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") " pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.168965 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:59:56.168944    2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 11 11:59:56.169022 ip-10-0-136-222 kubenswrapper[2562]: E0911 11:59:56.169011    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-webhook-cert podName:3c3385b3-c3fa-43b8-9b66-8db58f532a6d nodeName:}" failed. No retries permitted until 2026-09-11 11:59:56.66899178 +0000 UTC m=+505.169712046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-webhook-cert") pod "maas-controller-7db788bf7f-r5x4s" (UID: "3c3385b3-c3fa-43b8-9b66-8db58f532a6d") : secret "maas-controller-webhook-cert" not found
Sep 11 11:59:56.171265 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.171235    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-metrics-certs\") pod \"maas-controller-7db788bf7f-r5x4s\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") " pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.176469 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.176448    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcb5z\" (UniqueName: \"kubernetes.io/projected/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-kube-api-access-mcb5z\") pod \"maas-controller-7db788bf7f-r5x4s\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") " pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.472171 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.472140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-webhook-cert\") pod \"maas-controller-646d6b78cf-p9r7l\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") " pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:56.474484 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.474460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-webhook-cert\") pod \"maas-controller-646d6b78cf-p9r7l\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") " pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:56.674581 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.674543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-webhook-cert\") pod \"maas-controller-7db788bf7f-r5x4s\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") " pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.677013 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.676983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-webhook-cert\") pod \"maas-controller-7db788bf7f-r5x4s\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") " pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.682855 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.682836    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 11:59:56.865180 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.865145    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 11:59:56.953869 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:56.953755    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-646d6b78cf-p9r7l"]
Sep 11 11:59:56.956444 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:59:56.956420    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf59d37_6955_492e_a417_2376104532b8.slice/crio-a762a770b39377f2e688dd82a740cc58030d9bf1b94a7429c028ae788470e29c WatchSource:0}: Error finding container a762a770b39377f2e688dd82a740cc58030d9bf1b94a7429c028ae788470e29c: Status 404 returned error can't find the container with id a762a770b39377f2e688dd82a740cc58030d9bf1b94a7429c028ae788470e29c
Sep 11 11:59:57.136742 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:57.136657    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-r5x4s"]
Sep 11 11:59:57.139006 ip-10-0-136-222 kubenswrapper[2562]: W0911 11:59:57.138971    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3385b3_c3fa_43b8_9b66_8db58f532a6d.slice/crio-c897ccaeffadd9c60e28c3441a29c8b255d3c154b9785945e9e5199b340da770 WatchSource:0}: Error finding container c897ccaeffadd9c60e28c3441a29c8b255d3c154b9785945e9e5199b340da770: Status 404 returned error can't find the container with id c897ccaeffadd9c60e28c3441a29c8b255d3c154b9785945e9e5199b340da770
Sep 11 11:59:57.354373 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:57.354336    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7db788bf7f-r5x4s" event={"ID":"3c3385b3-c3fa-43b8-9b66-8db58f532a6d","Type":"ContainerStarted","Data":"c897ccaeffadd9c60e28c3441a29c8b255d3c154b9785945e9e5199b340da770"}
Sep 11 11:59:57.355499 ip-10-0-136-222 kubenswrapper[2562]: I0911 11:59:57.355465    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-646d6b78cf-p9r7l" event={"ID":"5bf59d37-6955-492e-a417-2376104532b8","Type":"ContainerStarted","Data":"a762a770b39377f2e688dd82a740cc58030d9bf1b94a7429c028ae788470e29c"}
Sep 11 12:00:00.374015 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:00.373965    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7db788bf7f-r5x4s" event={"ID":"3c3385b3-c3fa-43b8-9b66-8db58f532a6d","Type":"ContainerStarted","Data":"022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80"}
Sep 11 12:00:00.374516 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:00.374044    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 12:00:00.375531 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:00.375502    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-646d6b78cf-p9r7l" event={"ID":"5bf59d37-6955-492e-a417-2376104532b8","Type":"ContainerStarted","Data":"16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93"}
Sep 11 12:00:00.375661 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:00.375649    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 12:00:00.390776 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:00.390714    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7db788bf7f-r5x4s" podStartSLOduration=2.706189223 podStartE2EDuration="5.39069318s" totalImagesPullingTime="2.684503957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:00:00.389824062 +0000 UTC m=+508.890544332" watchObservedRunningTime="2026-09-11 12:00:00.39069318 +0000 UTC m=+508.891413452"
Sep 11 12:00:00.407986 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:00.407927    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-646d6b78cf-p9r7l" podStartSLOduration=2.547840976 podStartE2EDuration="5.407910135s" totalImagesPullingTime="2.860069159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:00:00.406948308 +0000 UTC m=+508.907668578" watchObservedRunningTime="2026-09-11 12:00:00.407910135 +0000 UTC m=+508.908630464"
Sep 11 12:00:11.384148 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.384115    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 12:00:11.384585 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.384190    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 12:00:11.443346 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.443294    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-646d6b78cf-p9r7l"]
Sep 11 12:00:11.443536 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.443510    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-646d6b78cf-p9r7l" podUID="5bf59d37-6955-492e-a417-2376104532b8" containerName="manager" containerID="cri-o://16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93" gracePeriod=30
Sep 11 12:00:11.692244 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.692213    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 12:00:11.817639 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.817599    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sfb2\" (UniqueName: \"kubernetes.io/projected/5bf59d37-6955-492e-a417-2376104532b8-kube-api-access-4sfb2\") pod \"5bf59d37-6955-492e-a417-2376104532b8\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") "
Sep 11 12:00:11.817806 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.817719    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-metrics-certs\") pod \"5bf59d37-6955-492e-a417-2376104532b8\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") "
Sep 11 12:00:11.817806 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.817766    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-webhook-cert\") pod \"5bf59d37-6955-492e-a417-2376104532b8\" (UID: \"5bf59d37-6955-492e-a417-2376104532b8\") "
Sep 11 12:00:11.819920 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.819879    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5bf59d37-6955-492e-a417-2376104532b8" (UID: "5bf59d37-6955-492e-a417-2376104532b8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:00:11.819920 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.819908    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5bf59d37-6955-492e-a417-2376104532b8" (UID: "5bf59d37-6955-492e-a417-2376104532b8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:00:11.819920 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.819892    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bf59d37-6955-492e-a417-2376104532b8-kube-api-access-4sfb2" (OuterVolumeSpecName: "kube-api-access-4sfb2") pod "5bf59d37-6955-492e-a417-2376104532b8" (UID: "5bf59d37-6955-492e-a417-2376104532b8"). InnerVolumeSpecName "kube-api-access-4sfb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:00:11.919186 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.919145    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sfb2\" (UniqueName: \"kubernetes.io/projected/5bf59d37-6955-492e-a417-2376104532b8-kube-api-access-4sfb2\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:00:11.919186 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.919175    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:00:11.919186 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:11.919185    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bf59d37-6955-492e-a417-2376104532b8-webhook-cert\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:00:12.433514 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:12.433478    2562 generic.go:356] "Generic (PLEG): container finished" podID="5bf59d37-6955-492e-a417-2376104532b8" containerID="16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93" exitCode=0
Sep 11 12:00:12.433514 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:12.433516    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-646d6b78cf-p9r7l" event={"ID":"5bf59d37-6955-492e-a417-2376104532b8","Type":"ContainerDied","Data":"16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93"}
Sep 11 12:00:12.433953 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:12.433524    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-646d6b78cf-p9r7l"
Sep 11 12:00:12.433953 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:12.433559    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-646d6b78cf-p9r7l" event={"ID":"5bf59d37-6955-492e-a417-2376104532b8","Type":"ContainerDied","Data":"a762a770b39377f2e688dd82a740cc58030d9bf1b94a7429c028ae788470e29c"}
Sep 11 12:00:12.433953 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:12.433581    2562 scope.go:117] "RemoveContainer" containerID="16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93"
Sep 11 12:00:12.441658 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:12.441638    2562 scope.go:117] "RemoveContainer" containerID="16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93"
Sep 11 12:00:12.441923 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:00:12.441904    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93\": container with ID starting with 16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93 not found: ID does not exist" containerID="16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93"
Sep 11 12:00:12.441989 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:12.441929    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93"} err="failed to get container status \"16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93\": rpc error: code = NotFound desc = could not find container \"16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93\": container with ID starting with 16d9d25a6718128e6ec8e5ee6c464a6fdc6a4e7d37201fcbd695f9171d548f93 not found: ID does not exist"
Sep 11 12:00:12.448510 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:12.448482    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-646d6b78cf-p9r7l"]
Sep 11 12:00:12.454880 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:12.454853    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-646d6b78cf-p9r7l"]
Sep 11 12:00:14.024393 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:14.024362    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bf59d37-6955-492e-a417-2376104532b8" path="/var/lib/kubelet/pods/5bf59d37-6955-492e-a417-2376104532b8/volumes"
Sep 11 12:00:33.353494 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.353454    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"]
Sep 11 12:00:33.354059 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.353988    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf59d37-6955-492e-a417-2376104532b8" containerName="manager"
Sep 11 12:00:33.354059 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.354010    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf59d37-6955-492e-a417-2376104532b8" containerName="manager"
Sep 11 12:00:33.354180 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.354126    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bf59d37-6955-492e-a417-2376104532b8" containerName="manager"
Sep 11 12:00:33.360572 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.360538    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.362655 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.362627    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 12:00:33.362789 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.362628    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 12:00:33.362789 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.362703    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p9257\""
Sep 11 12:00:33.366952 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.366905    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"]
Sep 11 12:00:33.511324 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.511283    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.511513 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.511338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-metrics-certs\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.511513 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.511360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljlsz\" (UniqueName: \"kubernetes.io/projected/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-kube-api-access-ljlsz\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.511513 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.511483    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-openshift-service-ca\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.612542 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.612460    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-openshift-service-ca\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.612542 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.612524    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.612775 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.612555    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-metrics-certs\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.612775 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.612580    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljlsz\" (UniqueName: \"kubernetes.io/projected/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-kube-api-access-ljlsz\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.612775 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:00:33.612666    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 11 12:00:33.612775 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:00:33.612704    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 11 12:00:33.612775 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:00:33.612766    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls podName:d75989e8-0c4b-427b-9de7-c23cd9d5ce8e nodeName:}" failed. No retries permitted until 2026-09-11 12:00:34.11274197 +0000 UTC m=+542.613462238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls") pod "maas-api-75bf587cb6-fprbd" (UID: "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e") : secret "maas-api-serving-cert" not found
Sep 11 12:00:33.613034 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:00:33.612787    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-metrics-certs podName:d75989e8-0c4b-427b-9de7-c23cd9d5ce8e nodeName:}" failed. No retries permitted until 2026-09-11 12:00:34.112777877 +0000 UTC m=+542.613498124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-metrics-certs") pod "maas-api-75bf587cb6-fprbd" (UID: "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e") : secret "maas-api-metrics-tls" not found
Sep 11 12:00:33.613161 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.613143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-openshift-service-ca\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.621564 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.621534    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljlsz\" (UniqueName: \"kubernetes.io/projected/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-kube-api-access-ljlsz\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:33.891757 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.891678    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"]
Sep 11 12:00:33.895146 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.895130    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"
Sep 11 12:00:33.897425 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.897402    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-q4sl7\""
Sep 11 12:00:33.897544 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.897423    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 12:00:33.907199 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:33.907175    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"]
Sep 11 12:00:34.016091 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.016054    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf95d\" (UniqueName: \"kubernetes.io/projected/e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d-kube-api-access-kf95d\") pod \"payload-processing-5b54b7fbb7-4mnp2\" (UID: \"e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"
Sep 11 12:00:34.016282 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.016240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-4mnp2\" (UID: \"e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"
Sep 11 12:00:34.116823 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.116782    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-4mnp2\" (UID: \"e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"
Sep 11 12:00:34.117038 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.116843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf95d\" (UniqueName: \"kubernetes.io/projected/e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d-kube-api-access-kf95d\") pod \"payload-processing-5b54b7fbb7-4mnp2\" (UID: \"e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"
Sep 11 12:00:34.117038 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.116910    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:34.117038 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.116936    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-metrics-certs\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:34.117038 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:00:34.117022    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 11 12:00:34.117178 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:00:34.117092    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls podName:d75989e8-0c4b-427b-9de7-c23cd9d5ce8e nodeName:}" failed. No retries permitted until 2026-09-11 12:00:35.117073382 +0000 UTC m=+543.617793633 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls") pod "maas-api-75bf587cb6-fprbd" (UID: "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e") : secret "maas-api-serving-cert" not found
Sep 11 12:00:34.117676 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.117654    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-4mnp2\" (UID: \"e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"
Sep 11 12:00:34.119446 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.119427    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-metrics-certs\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:34.124234 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.124212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf95d\" (UniqueName: \"kubernetes.io/projected/e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d-kube-api-access-kf95d\") pod \"payload-processing-5b54b7fbb7-4mnp2\" (UID: \"e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"
Sep 11 12:00:34.206286 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.206247    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"
Sep 11 12:00:34.480697 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.480614    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"]
Sep 11 12:00:34.484755 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:00:34.484729    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode884b1d0_9c9b_4f91_86cc_d2d0a90dc70d.slice/crio-13044e0707e0f9d32933179addb435cd1dbeb78c9c9d93c9aca35b7ae047a4cc WatchSource:0}: Error finding container 13044e0707e0f9d32933179addb435cd1dbeb78c9c9d93c9aca35b7ae047a4cc: Status 404 returned error can't find the container with id 13044e0707e0f9d32933179addb435cd1dbeb78c9c9d93c9aca35b7ae047a4cc
Sep 11 12:00:34.538565 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:34.538530    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2" event={"ID":"e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d","Type":"ContainerStarted","Data":"13044e0707e0f9d32933179addb435cd1dbeb78c9c9d93c9aca35b7ae047a4cc"}
Sep 11 12:00:35.126247 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:35.126206    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:35.129167 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:35.129141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls\") pod \"maas-api-75bf587cb6-fprbd\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") " pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:35.172387 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:35.172339    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:35.468036 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:35.467721    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"]
Sep 11 12:00:35.471061 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:00:35.471032    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd75989e8_0c4b_427b_9de7_c23cd9d5ce8e.slice/crio-d3181b4c645bc6070c944386b099341841736e237e2d8edb23a99c182a2577c9 WatchSource:0}: Error finding container d3181b4c645bc6070c944386b099341841736e237e2d8edb23a99c182a2577c9: Status 404 returned error can't find the container with id d3181b4c645bc6070c944386b099341841736e237e2d8edb23a99c182a2577c9
Sep 11 12:00:35.551630 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:35.551589    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd" event={"ID":"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e","Type":"ContainerStarted","Data":"d3181b4c645bc6070c944386b099341841736e237e2d8edb23a99c182a2577c9"}
Sep 11 12:00:36.557818 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:36.557781    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2" event={"ID":"e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d","Type":"ContainerStarted","Data":"2f20d4999d6ff77ffac6860ab6fc6b01b7191b8acc639af2dba38c7c29367e5e"}
Sep 11 12:00:36.558260 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:36.557964    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"
Sep 11 12:00:36.574787 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:36.574739    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2" podStartSLOduration=1.997880054 podStartE2EDuration="3.574724713s" totalImagesPullingTime="1.576844659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:00:36.573201748 +0000 UTC m=+545.073922015" watchObservedRunningTime="2026-09-11 12:00:36.574724713 +0000 UTC m=+545.075445019"
Sep 11 12:00:37.564510 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:37.564422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd" event={"ID":"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e","Type":"ContainerStarted","Data":"cdd86b9918e01c3351ed1218985573e7e70dde4b1280bdac1c639b08475a11bf"}
Sep 11 12:00:37.564510 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:37.564499    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:37.582813 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:37.582758    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd" podStartSLOduration=2.840256012 podStartE2EDuration="4.582741333s" totalImagesPullingTime="1.742485321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:00:37.581733575 +0000 UTC m=+546.082453846" watchObservedRunningTime="2026-09-11 12:00:37.582741333 +0000 UTC m=+546.083461601"
Sep 11 12:00:43.574184 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:43.574154    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:00:47.567493 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:00:47.567456    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-4mnp2"
Sep 11 12:01:14.200288 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.200248    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"]
Sep 11 12:01:14.206698 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.206674    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.209760 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.209729    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 12:01:14.209915 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.209759    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 12:01:14.209915 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.209808    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7x4n7\""
Sep 11 12:01:14.209915 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.209836    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 12:01:14.214052 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.213978    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"]
Sep 11 12:01:14.286997 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.286958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.287180 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.287020    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/caabba6f-86d9-457f-9b19-5507e9eae073-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.287180 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.287067    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.287180 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.287112    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.287180 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.287159    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.287344 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.287212    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf688\" (UniqueName: \"kubernetes.io/projected/caabba6f-86d9-457f-9b19-5507e9eae073-kube-api-access-tf688\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.388508 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.388462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.388717 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.388531    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/caabba6f-86d9-457f-9b19-5507e9eae073-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.388717 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.388551    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.388717 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.388569    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.388717 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.388612    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.388717 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.388642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf688\" (UniqueName: \"kubernetes.io/projected/caabba6f-86d9-457f-9b19-5507e9eae073-kube-api-access-tf688\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.388981 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.388925    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.388981 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.388938    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.389075 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.389060    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.390997 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.390971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/caabba6f-86d9-457f-9b19-5507e9eae073-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.391230 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.391209    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/caabba6f-86d9-457f-9b19-5507e9eae073-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.397789 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.397765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf688\" (UniqueName: \"kubernetes.io/projected/caabba6f-86d9-457f-9b19-5507e9eae073-kube-api-access-tf688\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg8qb\" (UID: \"caabba6f-86d9-457f-9b19-5507e9eae073\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.520136 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.520046    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:14.805469 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:14.805380    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"]
Sep 11 12:01:14.809241 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:01:14.809204    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaabba6f_86d9_457f_9b19_5507e9eae073.slice/crio-f2b9ed751653d9e261e863a7b84fc36aa24d3da3bb4d4ec6891d877f3e12bd06 WatchSource:0}: Error finding container f2b9ed751653d9e261e863a7b84fc36aa24d3da3bb4d4ec6891d877f3e12bd06: Status 404 returned error can't find the container with id f2b9ed751653d9e261e863a7b84fc36aa24d3da3bb4d4ec6891d877f3e12bd06
Sep 11 12:01:15.750712 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:15.750678    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb" event={"ID":"caabba6f-86d9-457f-9b19-5507e9eae073","Type":"ContainerStarted","Data":"f2b9ed751653d9e261e863a7b84fc36aa24d3da3bb4d4ec6891d877f3e12bd06"}
Sep 11 12:01:16.762510 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:16.762470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb" event={"ID":"caabba6f-86d9-457f-9b19-5507e9eae073","Type":"ContainerStarted","Data":"d3e7f8044207a578a24a562a62ca14d0e312830ef654297e05e9f5892471bc25"}
Sep 11 12:01:16.798882 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:16.798819    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb" podStartSLOduration=1.839301538 podStartE2EDuration="2.798802968s" totalImagesPullingTime="959.50143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:01:16.79768363 +0000 UTC m=+585.298403898" watchObservedRunningTime="2026-09-11 12:01:16.798802968 +0000 UTC m=+585.299523235"
Sep 11 12:01:24.520462 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:24.520416    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:24.520836 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:24.520484    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:24.533369 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:24.533341    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:24.810367 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:24.810273    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg8qb"
Sep 11 12:01:25.904896 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:25.904858    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"]
Sep 11 12:01:25.908533 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:25.908510    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:25.911654 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:25.911544    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 12:01:25.921290 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:25.921262    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"]
Sep 11 12:01:25.996767 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:25.996728    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:25.996982 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:25.996787    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e9a7825-ad1c-42c6-ace4-b07c0985276e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:25.996982 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:25.996870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:25.996982 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:25.996905    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tvjc\" (UniqueName: \"kubernetes.io/projected/2e9a7825-ad1c-42c6-ace4-b07c0985276e-kube-api-access-9tvjc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:25.996982 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:25.996968    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:25.997158 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:25.996996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.098094 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.098058    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e9a7825-ad1c-42c6-ace4-b07c0985276e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.098271 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.098131    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.098271 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.098162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tvjc\" (UniqueName: \"kubernetes.io/projected/2e9a7825-ad1c-42c6-ace4-b07c0985276e-kube-api-access-9tvjc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.098271 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.098195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.098271 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.098216    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.098271 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.098240    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.098800 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.098744    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.098800 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.098766    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.098800 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.098785    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.100593 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.100564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e9a7825-ad1c-42c6-ace4-b07c0985276e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.100856 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.100840    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e9a7825-ad1c-42c6-ace4-b07c0985276e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.108295 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.108254    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tvjc\" (UniqueName: \"kubernetes.io/projected/2e9a7825-ad1c-42c6-ace4-b07c0985276e-kube-api-access-9tvjc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2f7dm\" (UID: \"2e9a7825-ad1c-42c6-ace4-b07c0985276e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.220282 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.220250    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:26.496142 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.496113    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"]
Sep 11 12:01:26.498609 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:01:26.498581    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e9a7825_ad1c_42c6_ace4_b07c0985276e.slice/crio-04c832e14fc8ba119c5e3fecb23ce6f16d8682311fd63d0bf3ede2589f0198b5 WatchSource:0}: Error finding container 04c832e14fc8ba119c5e3fecb23ce6f16d8682311fd63d0bf3ede2589f0198b5: Status 404 returned error can't find the container with id 04c832e14fc8ba119c5e3fecb23ce6f16d8682311fd63d0bf3ede2589f0198b5
Sep 11 12:01:26.810683 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.810592    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm" event={"ID":"2e9a7825-ad1c-42c6-ace4-b07c0985276e","Type":"ContainerStarted","Data":"ef6f16636b27b4b6696ca0b392852c4d201e989243a28578a5a66e0ddf0540a0"}
Sep 11 12:01:26.810683 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.810643    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm" event={"ID":"2e9a7825-ad1c-42c6-ace4-b07c0985276e","Type":"ContainerStarted","Data":"04c832e14fc8ba119c5e3fecb23ce6f16d8682311fd63d0bf3ede2589f0198b5"}
Sep 11 12:01:26.833788 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:26.833738    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm" podStartSLOduration=1.66480899 podStartE2EDuration="1.833719983s" totalImagesPullingTime="168.910993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:01:26.833130508 +0000 UTC m=+595.333850777" watchObservedRunningTime="2026-09-11 12:01:26.833719983 +0000 UTC m=+595.334440272"
Sep 11 12:01:36.220773 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:36.220720    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:36.223229 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:36.220862    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:36.233127 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:36.233103    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:01:36.874446 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:01:36.874414    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2f7dm"
Sep 11 12:03:49.659899 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.659860    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"]
Sep 11 12:03:49.664148 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.664125    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.666879 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.666855    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-34511f-serving-cert\""
Sep 11 12:03:49.672441 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.671844    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"]
Sep 11 12:03:49.718571 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.718532    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48661d42-238f-47ec-8c27-3d8b8b0ff8db-openshift-service-ca\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.718737 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.718702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-maas-api-tls\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.718783 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.718744    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-metrics-certs\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.718783 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.718761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz9j8\" (UniqueName: \"kubernetes.io/projected/48661d42-238f-47ec-8c27-3d8b8b0ff8db-kube-api-access-kz9j8\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.819290 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.819249    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-maas-api-tls\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.819478 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.819328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-metrics-certs\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.819478 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.819356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz9j8\" (UniqueName: \"kubernetes.io/projected/48661d42-238f-47ec-8c27-3d8b8b0ff8db-kube-api-access-kz9j8\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.819478 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.819395    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48661d42-238f-47ec-8c27-3d8b8b0ff8db-openshift-service-ca\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.819478 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:03:49.819433    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-serving-cert: secret "maas-api-e2e-inf-a-34511f-serving-cert" not found
Sep 11 12:03:49.819623 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:03:49.819506    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-maas-api-tls podName:48661d42-238f-47ec-8c27-3d8b8b0ff8db nodeName:}" failed. No retries permitted until 2026-09-11 12:03:50.319484016 +0000 UTC m=+738.820204262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-maas-api-tls") pod "maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm" (UID: "48661d42-238f-47ec-8c27-3d8b8b0ff8db") : secret "maas-api-e2e-inf-a-34511f-serving-cert" not found
Sep 11 12:03:49.820168 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.820135    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48661d42-238f-47ec-8c27-3d8b8b0ff8db-openshift-service-ca\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.821866 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.821845    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-metrics-certs\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:49.831088 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:49.831068    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz9j8\" (UniqueName: \"kubernetes.io/projected/48661d42-238f-47ec-8c27-3d8b8b0ff8db-kube-api-access-kz9j8\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:50.324653 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.324614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-maas-api-tls\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:50.327045 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.327023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-maas-api-tls\") pod \"maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:50.427238 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.427200    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"]
Sep 11 12:03:50.430986 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.430962    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:03:50.433284 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.433262    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-34511f\""
Sep 11 12:03:50.433436 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.433335    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-34511f-dockercfg-s8hf8\""
Sep 11 12:03:50.439418 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.439390    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"]
Sep 11 12:03:50.525942 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.525901    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87w58\" (UniqueName: \"kubernetes.io/projected/fa5c5396-81ea-4410-b5da-102c1281bc43-kube-api-access-87w58\") pod \"payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k\" (UID: \"fa5c5396-81ea-4410-b5da-102c1281bc43\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:03:50.526128 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.526031    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5c5396-81ea-4410-b5da-102c1281bc43-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k\" (UID: \"fa5c5396-81ea-4410-b5da-102c1281bc43\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:03:50.577649 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.577551    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:50.626937 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.626898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5c5396-81ea-4410-b5da-102c1281bc43-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k\" (UID: \"fa5c5396-81ea-4410-b5da-102c1281bc43\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:03:50.627439 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.627411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87w58\" (UniqueName: \"kubernetes.io/projected/fa5c5396-81ea-4410-b5da-102c1281bc43-kube-api-access-87w58\") pod \"payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k\" (UID: \"fa5c5396-81ea-4410-b5da-102c1281bc43\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:03:50.628253 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.628077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5c5396-81ea-4410-b5da-102c1281bc43-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k\" (UID: \"fa5c5396-81ea-4410-b5da-102c1281bc43\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:03:50.635469 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.635446    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87w58\" (UniqueName: \"kubernetes.io/projected/fa5c5396-81ea-4410-b5da-102c1281bc43-kube-api-access-87w58\") pod \"payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k\" (UID: \"fa5c5396-81ea-4410-b5da-102c1281bc43\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:03:50.742631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.742603    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:03:50.854249 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.854214    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"]
Sep 11 12:03:50.857258 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:03:50.857226    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48661d42_238f_47ec_8c27_3d8b8b0ff8db.slice/crio-a49ad0153f62ea9381647fe215bbfe1c58761e0cbd498b0127fca41b8584d965 WatchSource:0}: Error finding container a49ad0153f62ea9381647fe215bbfe1c58761e0cbd498b0127fca41b8584d965: Status 404 returned error can't find the container with id a49ad0153f62ea9381647fe215bbfe1c58761e0cbd498b0127fca41b8584d965
Sep 11 12:03:50.859008 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:50.858984    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:03:51.018534 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:51.018392    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"]
Sep 11 12:03:51.020743 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:03:51.020716    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa5c5396_81ea_4410_b5da_102c1281bc43.slice/crio-6c98f92046451c73199e74738fa2d5e5438451ff7822a419da54723a2af1c44b WatchSource:0}: Error finding container 6c98f92046451c73199e74738fa2d5e5438451ff7822a419da54723a2af1c44b: Status 404 returned error can't find the container with id 6c98f92046451c73199e74738fa2d5e5438451ff7822a419da54723a2af1c44b
Sep 11 12:03:51.491071 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:51.491030    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm" event={"ID":"48661d42-238f-47ec-8c27-3d8b8b0ff8db","Type":"ContainerStarted","Data":"a49ad0153f62ea9381647fe215bbfe1c58761e0cbd498b0127fca41b8584d965"}
Sep 11 12:03:51.493056 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:51.493024    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k" event={"ID":"fa5c5396-81ea-4410-b5da-102c1281bc43","Type":"ContainerStarted","Data":"2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7"}
Sep 11 12:03:51.493222 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:51.493083    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k" event={"ID":"fa5c5396-81ea-4410-b5da-102c1281bc43","Type":"ContainerStarted","Data":"6c98f92046451c73199e74738fa2d5e5438451ff7822a419da54723a2af1c44b"}
Sep 11 12:03:51.493222 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:51.493189    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:03:51.508513 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:51.508458    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k" podStartSLOduration=1.5084419759999999 podStartE2EDuration="1.508441976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:51.506994316 +0000 UTC m=+740.007714583" watchObservedRunningTime="2026-09-11 12:03:51.508441976 +0000 UTC m=+740.009162243"
Sep 11 12:03:52.498936 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:52.498901    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm" event={"ID":"48661d42-238f-47ec-8c27-3d8b8b0ff8db","Type":"ContainerStarted","Data":"115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8"}
Sep 11 12:03:52.515135 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:52.515079    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm" podStartSLOduration=2.452840574 podStartE2EDuration="3.515060331s" totalImagesPullingTime="1.062219757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:52.514440071 +0000 UTC m=+741.015160336" watchObservedRunningTime="2026-09-11 12:03:52.515060331 +0000 UTC m=+741.015780598"
Sep 11 12:03:53.505363 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:53.505326    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:57.147383 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.147350    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"]
Sep 11 12:03:57.156856 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.156829    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.159505 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.159480    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a9aab05d-serving-cert\""
Sep 11 12:03:57.161423 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.161395    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"]
Sep 11 12:03:57.290559 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.290526    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-metrics-certs\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.290777 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.290584    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29h8c\" (UniqueName: \"kubernetes.io/projected/e9e7bf53-9c68-40ba-b10c-688df2219c97-kube-api-access-29h8c\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.290777 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.290699    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-maas-api-tls\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.290777 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.290742    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e7bf53-9c68-40ba-b10c-688df2219c97-openshift-service-ca\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.391326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.391265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-metrics-certs\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.391524 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.391367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29h8c\" (UniqueName: \"kubernetes.io/projected/e9e7bf53-9c68-40ba-b10c-688df2219c97-kube-api-access-29h8c\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.391524 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.391430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-maas-api-tls\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.391524 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.391461    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e7bf53-9c68-40ba-b10c-688df2219c97-openshift-service-ca\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.392190 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.392163    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e7bf53-9c68-40ba-b10c-688df2219c97-openshift-service-ca\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.394103 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.394043    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-metrics-certs\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.394507 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.394483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-maas-api-tls\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.399057 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.398998    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29h8c\" (UniqueName: \"kubernetes.io/projected/e9e7bf53-9c68-40ba-b10c-688df2219c97-kube-api-access-29h8c\") pod \"maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.471832 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.471794    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:57.753237 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:57.753193    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"]
Sep 11 12:03:57.760533 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:03:57.760502    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e7bf53_9c68_40ba_b10c_688df2219c97.slice/crio-30e9186efdb7a6ba1af4d71a4923922e53af3c94138071f5495c0a0fe0bd6a66 WatchSource:0}: Error finding container 30e9186efdb7a6ba1af4d71a4923922e53af3c94138071f5495c0a0fe0bd6a66: Status 404 returned error can't find the container with id 30e9186efdb7a6ba1af4d71a4923922e53af3c94138071f5495c0a0fe0bd6a66
Sep 11 12:03:58.084979 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.084895    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"]
Sep 11 12:03:58.089537 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.089518    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:03:58.091512 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.091482    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a9aab05d\""
Sep 11 12:03:58.091638 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.091529    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a9aab05d-dockercfg-hdshc\""
Sep 11 12:03:58.097738 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.097712    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"]
Sep 11 12:03:58.199350 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.199316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c924add4-ccf5-44a1-b689-a50ba30dad73-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n\" (UID: \"c924add4-ccf5-44a1-b689-a50ba30dad73\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:03:58.199714 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.199364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdjcj\" (UniqueName: \"kubernetes.io/projected/c924add4-ccf5-44a1-b689-a50ba30dad73-kube-api-access-gdjcj\") pod \"payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n\" (UID: \"c924add4-ccf5-44a1-b689-a50ba30dad73\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:03:58.300980 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.300937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c924add4-ccf5-44a1-b689-a50ba30dad73-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n\" (UID: \"c924add4-ccf5-44a1-b689-a50ba30dad73\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:03:58.301139 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.300994    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdjcj\" (UniqueName: \"kubernetes.io/projected/c924add4-ccf5-44a1-b689-a50ba30dad73-kube-api-access-gdjcj\") pod \"payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n\" (UID: \"c924add4-ccf5-44a1-b689-a50ba30dad73\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:03:58.301715 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.301694    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c924add4-ccf5-44a1-b689-a50ba30dad73-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n\" (UID: \"c924add4-ccf5-44a1-b689-a50ba30dad73\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:03:58.308733 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.308704    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdjcj\" (UniqueName: \"kubernetes.io/projected/c924add4-ccf5-44a1-b689-a50ba30dad73-kube-api-access-gdjcj\") pod \"payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n\" (UID: \"c924add4-ccf5-44a1-b689-a50ba30dad73\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:03:58.402788 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.402707    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:03:58.531065 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.531016    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6" event={"ID":"e9e7bf53-9c68-40ba-b10c-688df2219c97","Type":"ContainerStarted","Data":"30e9186efdb7a6ba1af4d71a4923922e53af3c94138071f5495c0a0fe0bd6a66"}
Sep 11 12:03:58.678033 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.677812    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"]
Sep 11 12:03:58.680650 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:03:58.680621    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc924add4_ccf5_44a1_b689_a50ba30dad73.slice/crio-6faa8801f7066895e67494563c14026d181ec0fb70556a1207eb96b0adc23c07 WatchSource:0}: Error finding container 6faa8801f7066895e67494563c14026d181ec0fb70556a1207eb96b0adc23c07: Status 404 returned error can't find the container with id 6faa8801f7066895e67494563c14026d181ec0fb70556a1207eb96b0adc23c07
Sep 11 12:03:58.832114 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.832074    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"]
Sep 11 12:03:58.836280 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.836254    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:58.838212 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.838189    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-2c2847-serving-cert\""
Sep 11 12:03:58.846351 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.846289    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"]
Sep 11 12:03:58.906108 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.906068    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv7b5\" (UniqueName: \"kubernetes.io/projected/eb6304ea-12a8-484b-be5d-c074e1020907-kube-api-access-pv7b5\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:58.906255 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.906131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-metrics-certs\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:58.906255 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.906217    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb6304ea-12a8-484b-be5d-c074e1020907-openshift-service-ca\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:58.906616 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:58.906260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-maas-api-tls\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:59.007845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.007747    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv7b5\" (UniqueName: \"kubernetes.io/projected/eb6304ea-12a8-484b-be5d-c074e1020907-kube-api-access-pv7b5\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:59.007845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.007802    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-metrics-certs\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:59.007845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.007825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb6304ea-12a8-484b-be5d-c074e1020907-openshift-service-ca\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:59.008120 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.007849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-maas-api-tls\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:59.008568 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.008541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb6304ea-12a8-484b-be5d-c074e1020907-openshift-service-ca\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:59.010386 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.010360    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-maas-api-tls\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:59.010541 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.010520    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-metrics-certs\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:59.015822 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.015793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv7b5\" (UniqueName: \"kubernetes.io/projected/eb6304ea-12a8-484b-be5d-c074e1020907-kube-api-access-pv7b5\") pod \"maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:59.149887 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.149849    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:03:59.432409 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.432374    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"]
Sep 11 12:03:59.434717 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:03:59.434676    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb6304ea_12a8_484b_be5d_c074e1020907.slice/crio-3429cf4c31bcd74ff378c7db528b6c91e17133c898db8327ee008acb00d9cfd7 WatchSource:0}: Error finding container 3429cf4c31bcd74ff378c7db528b6c91e17133c898db8327ee008acb00d9cfd7: Status 404 returned error can't find the container with id 3429cf4c31bcd74ff378c7db528b6c91e17133c898db8327ee008acb00d9cfd7
Sep 11 12:03:59.515370 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.515336    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:03:59.539080 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.539042    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6" event={"ID":"e9e7bf53-9c68-40ba-b10c-688df2219c97","Type":"ContainerStarted","Data":"2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452"}
Sep 11 12:03:59.539286 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.539107    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:03:59.540617 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.540584    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb" event={"ID":"eb6304ea-12a8-484b-be5d-c074e1020907","Type":"ContainerStarted","Data":"3429cf4c31bcd74ff378c7db528b6c91e17133c898db8327ee008acb00d9cfd7"}
Sep 11 12:03:59.542451 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.542421    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n" event={"ID":"c924add4-ccf5-44a1-b689-a50ba30dad73","Type":"ContainerStarted","Data":"7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8"}
Sep 11 12:03:59.542566 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.542478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n" event={"ID":"c924add4-ccf5-44a1-b689-a50ba30dad73","Type":"ContainerStarted","Data":"6faa8801f7066895e67494563c14026d181ec0fb70556a1207eb96b0adc23c07"}
Sep 11 12:03:59.542625 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.542581    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:03:59.562755 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.562685    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6" podStartSLOduration=1.632466382 podStartE2EDuration="2.562661735s" totalImagesPullingTime="930.195353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:59.559950427 +0000 UTC m=+748.060670695" watchObservedRunningTime="2026-09-11 12:03:59.562661735 +0000 UTC m=+748.063382004"
Sep 11 12:03:59.599210 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.598986    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n" podStartSLOduration=1.5989701269999999 podStartE2EDuration="1.598970127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:03:59.598508046 +0000 UTC m=+748.099228314" watchObservedRunningTime="2026-09-11 12:03:59.598970127 +0000 UTC m=+748.099690395"
Sep 11 12:03:59.630321 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.630283    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"]
Sep 11 12:03:59.634986 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.634968    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:03:59.637778 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.637760    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-2c2847\""
Sep 11 12:03:59.637855 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.637823    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-2c2847-dockercfg-mtkdd\""
Sep 11 12:03:59.653673 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.653640    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"]
Sep 11 12:03:59.717109 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.716876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b752d36b-1b53-4f5c-9995-dbbc28e7c171-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn\" (UID: \"b752d36b-1b53-4f5c-9995-dbbc28e7c171\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:03:59.717109 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.716943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frkwt\" (UniqueName: \"kubernetes.io/projected/b752d36b-1b53-4f5c-9995-dbbc28e7c171-kube-api-access-frkwt\") pod \"payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn\" (UID: \"b752d36b-1b53-4f5c-9995-dbbc28e7c171\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:03:59.817572 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.817532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b752d36b-1b53-4f5c-9995-dbbc28e7c171-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn\" (UID: \"b752d36b-1b53-4f5c-9995-dbbc28e7c171\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:03:59.817572 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.817573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frkwt\" (UniqueName: \"kubernetes.io/projected/b752d36b-1b53-4f5c-9995-dbbc28e7c171-kube-api-access-frkwt\") pod \"payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn\" (UID: \"b752d36b-1b53-4f5c-9995-dbbc28e7c171\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:03:59.818582 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.818558    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b752d36b-1b53-4f5c-9995-dbbc28e7c171-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn\" (UID: \"b752d36b-1b53-4f5c-9995-dbbc28e7c171\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:03:59.830693 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.830666    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frkwt\" (UniqueName: \"kubernetes.io/projected/b752d36b-1b53-4f5c-9995-dbbc28e7c171-kube-api-access-frkwt\") pod \"payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn\" (UID: \"b752d36b-1b53-4f5c-9995-dbbc28e7c171\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:03:59.945090 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:03:59.945063    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:04:00.226057 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:00.226032    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"]
Sep 11 12:04:00.230192 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:04:00.230164    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb752d36b_1b53_4f5c_9995_dbbc28e7c171.slice/crio-5f3042d4daf38242563fdd4ac781d0e51f2de352806e809679f0372c48f624e4 WatchSource:0}: Error finding container 5f3042d4daf38242563fdd4ac781d0e51f2de352806e809679f0372c48f624e4: Status 404 returned error can't find the container with id 5f3042d4daf38242563fdd4ac781d0e51f2de352806e809679f0372c48f624e4
Sep 11 12:04:00.558238 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:00.558147    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn" event={"ID":"b752d36b-1b53-4f5c-9995-dbbc28e7c171","Type":"ContainerStarted","Data":"f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3"}
Sep 11 12:04:00.558698 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:00.558236    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:04:00.558698 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:00.558264    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn" event={"ID":"b752d36b-1b53-4f5c-9995-dbbc28e7c171","Type":"ContainerStarted","Data":"5f3042d4daf38242563fdd4ac781d0e51f2de352806e809679f0372c48f624e4"}
Sep 11 12:04:00.559761 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:00.559731    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb" event={"ID":"eb6304ea-12a8-484b-be5d-c074e1020907","Type":"ContainerStarted","Data":"90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82"}
Sep 11 12:04:00.559957 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:00.559938    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:04:00.573582 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:00.573521    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn" podStartSLOduration=1.5735009359999999 podStartE2EDuration="1.573500936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:00.572168457 +0000 UTC m=+749.072888725" watchObservedRunningTime="2026-09-11 12:04:00.573500936 +0000 UTC m=+749.074221206"
Sep 11 12:04:00.590105 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:00.590044    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb" podStartSLOduration=2.119211597 podStartE2EDuration="2.590025082s" totalImagesPullingTime="470.813485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:00.589928959 +0000 UTC m=+749.090649233" watchObservedRunningTime="2026-09-11 12:04:00.590025082 +0000 UTC m=+749.090745349"
Sep 11 12:04:02.501225 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:02.501188    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:04:05.565753 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:05.565722    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:04:06.571098 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:06.571068    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:04:10.561888 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:10.561854    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:04:11.568217 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:11.568180    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:04:17.849520 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:17.849480    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"]
Sep 11 12:04:17.855226 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:17.855202    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:17.857849 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:17.857828    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e42660-serving-cert\""
Sep 11 12:04:17.864829 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:17.864798    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"]
Sep 11 12:04:17.898576 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:17.898536    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:17.898772 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:17.898587    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqp8z\" (UniqueName: \"kubernetes.io/projected/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-kube-api-access-nqp8z\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:17.898772 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:17.898609    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-metrics-certs\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:17.898772 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:17.898673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-maas-api-tls\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:18.000269 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:18.000224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:18.000497 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:18.000279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqp8z\" (UniqueName: \"kubernetes.io/projected/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-kube-api-access-nqp8z\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:18.000497 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:18.000324    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-metrics-certs\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:18.000497 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:18.000371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-maas-api-tls\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:18.000627 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:04:18.000565    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-serving-cert: secret "maas-api-e2e-inf-b-e42660-serving-cert" not found
Sep 11 12:04:18.000675 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:04:18.000659    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-maas-api-tls podName:2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4 nodeName:}" failed. No retries permitted until 2026-09-11 12:04:18.500636768 +0000 UTC m=+767.001357019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-maas-api-tls") pod "maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5" (UID: "2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4") : secret "maas-api-e2e-inf-b-e42660-serving-cert" not found
Sep 11 12:04:18.001036 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:18.001019    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:18.002821 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:18.002795    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-metrics-certs\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:18.009344 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:18.009298    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqp8z\" (UniqueName: \"kubernetes.io/projected/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-kube-api-access-nqp8z\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:18.504900 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:18.504865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-maas-api-tls\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:18.507219 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:18.507195    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-maas-api-tls\") pod \"maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:18.768848 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:18.768774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:19.057875 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:19.057839    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"]
Sep 11 12:04:19.060692 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:04:19.060660    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd4b6b5_481a_4fdc_b0ac_4b52b7fe43f4.slice/crio-a7b7ab6dc648bcf8d29c611323fed46c5a16e14f3c4dc735d25c7c2d8585d044 WatchSource:0}: Error finding container a7b7ab6dc648bcf8d29c611323fed46c5a16e14f3c4dc735d25c7c2d8585d044: Status 404 returned error can't find the container with id a7b7ab6dc648bcf8d29c611323fed46c5a16e14f3c4dc735d25c7c2d8585d044
Sep 11 12:04:19.662612 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:19.662579    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5" event={"ID":"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4","Type":"ContainerStarted","Data":"a7b7ab6dc648bcf8d29c611323fed46c5a16e14f3c4dc735d25c7c2d8585d044"}
Sep 11 12:04:20.669531 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:20.669491    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5" event={"ID":"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4","Type":"ContainerStarted","Data":"1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474"}
Sep 11 12:04:20.669971 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:20.669725    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:20.687737 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:20.687686    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5" podStartSLOduration=2.845228096 podStartE2EDuration="3.687671088s" totalImagesPullingTime="842.442992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:20.685415174 +0000 UTC m=+769.186135443" watchObservedRunningTime="2026-09-11 12:04:20.687671088 +0000 UTC m=+769.188391355"
Sep 11 12:04:26.680629 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:26.680595    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:04:38.064285 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.064253    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"]
Sep 11 12:04:38.064883 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.064478    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6" podUID="e9e7bf53-9c68-40ba-b10c-688df2219c97" containerName="maas-api" containerID="cri-o://2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452" gracePeriod=30
Sep 11 12:04:38.129227 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.129191    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"]
Sep 11 12:04:38.129486 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.129442    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n" podUID="c924add4-ccf5-44a1-b689-a50ba30dad73" containerName="payload-pre-processing" containerID="cri-o://7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8" gracePeriod=30
Sep 11 12:04:38.379006 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.378972    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:04:38.418175 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.418144    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:04:38.499557 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.499523    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e7bf53-9c68-40ba-b10c-688df2219c97-openshift-service-ca\") pod \"e9e7bf53-9c68-40ba-b10c-688df2219c97\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") "
Sep 11 12:04:38.499736 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.499594    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-metrics-certs\") pod \"e9e7bf53-9c68-40ba-b10c-688df2219c97\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") "
Sep 11 12:04:38.499736 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.499620    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdjcj\" (UniqueName: \"kubernetes.io/projected/c924add4-ccf5-44a1-b689-a50ba30dad73-kube-api-access-gdjcj\") pod \"c924add4-ccf5-44a1-b689-a50ba30dad73\" (UID: \"c924add4-ccf5-44a1-b689-a50ba30dad73\") "
Sep 11 12:04:38.499736 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.499639    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c924add4-ccf5-44a1-b689-a50ba30dad73-plugins-config-volume\") pod \"c924add4-ccf5-44a1-b689-a50ba30dad73\" (UID: \"c924add4-ccf5-44a1-b689-a50ba30dad73\") "
Sep 11 12:04:38.499736 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.499694    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29h8c\" (UniqueName: \"kubernetes.io/projected/e9e7bf53-9c68-40ba-b10c-688df2219c97-kube-api-access-29h8c\") pod \"e9e7bf53-9c68-40ba-b10c-688df2219c97\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") "
Sep 11 12:04:38.499939 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.499761    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-maas-api-tls\") pod \"e9e7bf53-9c68-40ba-b10c-688df2219c97\" (UID: \"e9e7bf53-9c68-40ba-b10c-688df2219c97\") "
Sep 11 12:04:38.500060 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.500013    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c924add4-ccf5-44a1-b689-a50ba30dad73-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c924add4-ccf5-44a1-b689-a50ba30dad73" (UID: "c924add4-ccf5-44a1-b689-a50ba30dad73"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:38.500060 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.500009    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9e7bf53-9c68-40ba-b10c-688df2219c97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9e7bf53-9c68-40ba-b10c-688df2219c97" (UID: "e9e7bf53-9c68-40ba-b10c-688df2219c97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:38.500243 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.500136    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e7bf53-9c68-40ba-b10c-688df2219c97-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:04:38.500243 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.500151    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c924add4-ccf5-44a1-b689-a50ba30dad73-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:04:38.502058 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.502035    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e7bf53-9c68-40ba-b10c-688df2219c97-kube-api-access-29h8c" (OuterVolumeSpecName: "kube-api-access-29h8c") pod "e9e7bf53-9c68-40ba-b10c-688df2219c97" (UID: "e9e7bf53-9c68-40ba-b10c-688df2219c97"). InnerVolumeSpecName "kube-api-access-29h8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:38.502156 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.502072    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c924add4-ccf5-44a1-b689-a50ba30dad73-kube-api-access-gdjcj" (OuterVolumeSpecName: "kube-api-access-gdjcj") pod "c924add4-ccf5-44a1-b689-a50ba30dad73" (UID: "c924add4-ccf5-44a1-b689-a50ba30dad73"). InnerVolumeSpecName "kube-api-access-gdjcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:38.502224 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.502199    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9e7bf53-9c68-40ba-b10c-688df2219c97" (UID: "e9e7bf53-9c68-40ba-b10c-688df2219c97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:04:38.502259 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.502209    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e9e7bf53-9c68-40ba-b10c-688df2219c97" (UID: "e9e7bf53-9c68-40ba-b10c-688df2219c97"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:04:38.601270 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.601186    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29h8c\" (UniqueName: \"kubernetes.io/projected/e9e7bf53-9c68-40ba-b10c-688df2219c97-kube-api-access-29h8c\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:04:38.601270 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.601222    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:04:38.601270 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.601231    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e7bf53-9c68-40ba-b10c-688df2219c97-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:04:38.601270 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.601241    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdjcj\" (UniqueName: \"kubernetes.io/projected/c924add4-ccf5-44a1-b689-a50ba30dad73-kube-api-access-gdjcj\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:04:38.759898 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.759867    2562 generic.go:356] "Generic (PLEG): container finished" podID="e9e7bf53-9c68-40ba-b10c-688df2219c97" containerID="2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452" exitCode=0
Sep 11 12:04:38.760075 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.759920    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"
Sep 11 12:04:38.760075 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.759965    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6" event={"ID":"e9e7bf53-9c68-40ba-b10c-688df2219c97","Type":"ContainerDied","Data":"2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452"}
Sep 11 12:04:38.760075 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.760021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6" event={"ID":"e9e7bf53-9c68-40ba-b10c-688df2219c97","Type":"ContainerDied","Data":"30e9186efdb7a6ba1af4d71a4923922e53af3c94138071f5495c0a0fe0bd6a66"}
Sep 11 12:04:38.760075 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.760051    2562 scope.go:117] "RemoveContainer" containerID="2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452"
Sep 11 12:04:38.761604 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.761581    2562 generic.go:356] "Generic (PLEG): container finished" podID="c924add4-ccf5-44a1-b689-a50ba30dad73" containerID="7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8" exitCode=0
Sep 11 12:04:38.761728 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.761621    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n" event={"ID":"c924add4-ccf5-44a1-b689-a50ba30dad73","Type":"ContainerDied","Data":"7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8"}
Sep 11 12:04:38.761728 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.761629    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"
Sep 11 12:04:38.761728 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.761662    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n" event={"ID":"c924add4-ccf5-44a1-b689-a50ba30dad73","Type":"ContainerDied","Data":"6faa8801f7066895e67494563c14026d181ec0fb70556a1207eb96b0adc23c07"}
Sep 11 12:04:38.769754 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.769736    2562 scope.go:117] "RemoveContainer" containerID="2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452"
Sep 11 12:04:38.770020 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:04:38.770000    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452\": container with ID starting with 2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452 not found: ID does not exist" containerID="2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452"
Sep 11 12:04:38.770084 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.770027    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452"} err="failed to get container status \"2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452\": rpc error: code = NotFound desc = could not find container \"2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452\": container with ID starting with 2ef8d6273e0ae82ff5bd2cb09df99277bfb7aefc47b0c5e132752b716a0e2452 not found: ID does not exist"
Sep 11 12:04:38.770084 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.770044    2562 scope.go:117] "RemoveContainer" containerID="7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8"
Sep 11 12:04:38.778158 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.778138    2562 scope.go:117] "RemoveContainer" containerID="7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8"
Sep 11 12:04:38.778469 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:04:38.778450    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8\": container with ID starting with 7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8 not found: ID does not exist" containerID="7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8"
Sep 11 12:04:38.778530 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.778475    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8"} err="failed to get container status \"7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8\": rpc error: code = NotFound desc = could not find container \"7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8\": container with ID starting with 7326d5e7690ce0edac99d00d62b28e742fa5942aa6cd4e275f573cb3453ff9e8 not found: ID does not exist"
Sep 11 12:04:38.782715 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.782695    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"]
Sep 11 12:04:38.791103 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.791081    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9aab05d-7dd975d794-7hcx6"]
Sep 11 12:04:38.800478 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.800456    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"]
Sep 11 12:04:38.804538 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:38.804516    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9aab05d-585c5c7974-jkb7n"]
Sep 11 12:04:40.025405 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:40.025367    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c924add4-ccf5-44a1-b689-a50ba30dad73" path="/var/lib/kubelet/pods/c924add4-ccf5-44a1-b689-a50ba30dad73/volumes"
Sep 11 12:04:40.025846 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:40.025832    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9e7bf53-9c68-40ba-b10c-688df2219c97" path="/var/lib/kubelet/pods/e9e7bf53-9c68-40ba-b10c-688df2219c97/volumes"
Sep 11 12:04:42.727728 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.727691    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"]
Sep 11 12:04:42.728161 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.728147    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9e7bf53-9c68-40ba-b10c-688df2219c97" containerName="maas-api"
Sep 11 12:04:42.728224 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.728165    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9e7bf53-9c68-40ba-b10c-688df2219c97" containerName="maas-api"
Sep 11 12:04:42.728224 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.728215    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c924add4-ccf5-44a1-b689-a50ba30dad73" containerName="payload-pre-processing"
Sep 11 12:04:42.728338 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.728225    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c924add4-ccf5-44a1-b689-a50ba30dad73" containerName="payload-pre-processing"
Sep 11 12:04:42.728338 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.728329    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9e7bf53-9c68-40ba-b10c-688df2219c97" containerName="maas-api"
Sep 11 12:04:42.728338 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.728339    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c924add4-ccf5-44a1-b689-a50ba30dad73" containerName="payload-pre-processing"
Sep 11 12:04:42.733154 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.733131    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.735207 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.735179    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-a6d3d15b-serving-cert\""
Sep 11 12:04:42.744169 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.744144    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"]
Sep 11 12:04:42.842443 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.842407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhc7k\" (UniqueName: \"kubernetes.io/projected/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-kube-api-access-jhc7k\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.842616 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.842489    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-openshift-service-ca\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.842616 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.842515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-maas-api-tls\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.842616 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.842535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-metrics-certs\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.943180 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.943137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-openshift-service-ca\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.943426 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.943185    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-maas-api-tls\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.943426 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.943343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-metrics-certs\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.943546 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.943491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhc7k\" (UniqueName: \"kubernetes.io/projected/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-kube-api-access-jhc7k\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.943895 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.943864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-openshift-service-ca\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.945671 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.945643    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-maas-api-tls\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.945788 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.945770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-metrics-certs\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:42.951613 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:42.951590    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhc7k\" (UniqueName: \"kubernetes.io/projected/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-kube-api-access-jhc7k\") pod \"maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:43.046549 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.046463    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:43.319337 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.319225    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"]
Sep 11 12:04:43.321876 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:04:43.321849    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce10dd5_4a81_448b_8a9a_06d0a26fb27b.slice/crio-62321973c4d23288f983661ebd07c1f6c11ab5dd5fe2583b00696442d81e989d WatchSource:0}: Error finding container 62321973c4d23288f983661ebd07c1f6c11ab5dd5fe2583b00696442d81e989d: Status 404 returned error can't find the container with id 62321973c4d23288f983661ebd07c1f6c11ab5dd5fe2583b00696442d81e989d
Sep 11 12:04:43.365587 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.365552    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"]
Sep 11 12:04:43.370113 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.370098    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:04:43.372122 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.372104    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-a6d3d15b\""
Sep 11 12:04:43.372203 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.372168    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-a6d3d15b-dockercfg-zfjk4\""
Sep 11 12:04:43.381767 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.381743    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"]
Sep 11 12:04:43.549702 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.549667    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x58lk\" (UniqueName: \"kubernetes.io/projected/07e8692a-bc60-4a8d-9dfc-44f347ce2145-kube-api-access-x58lk\") pod \"payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255\" (UID: \"07e8692a-bc60-4a8d-9dfc-44f347ce2145\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:04:43.549702 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.549705    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07e8692a-bc60-4a8d-9dfc-44f347ce2145-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255\" (UID: \"07e8692a-bc60-4a8d-9dfc-44f347ce2145\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:04:43.650778 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.650685    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x58lk\" (UniqueName: \"kubernetes.io/projected/07e8692a-bc60-4a8d-9dfc-44f347ce2145-kube-api-access-x58lk\") pod \"payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255\" (UID: \"07e8692a-bc60-4a8d-9dfc-44f347ce2145\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:04:43.650778 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.650729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07e8692a-bc60-4a8d-9dfc-44f347ce2145-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255\" (UID: \"07e8692a-bc60-4a8d-9dfc-44f347ce2145\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:04:43.651513 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.651494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07e8692a-bc60-4a8d-9dfc-44f347ce2145-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255\" (UID: \"07e8692a-bc60-4a8d-9dfc-44f347ce2145\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:04:43.658546 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.658525    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x58lk\" (UniqueName: \"kubernetes.io/projected/07e8692a-bc60-4a8d-9dfc-44f347ce2145-kube-api-access-x58lk\") pod \"payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255\" (UID: \"07e8692a-bc60-4a8d-9dfc-44f347ce2145\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:04:43.682492 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.682462    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:04:43.791745 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.791709    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp" event={"ID":"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b","Type":"ContainerStarted","Data":"62321973c4d23288f983661ebd07c1f6c11ab5dd5fe2583b00696442d81e989d"}
Sep 11 12:04:43.957548 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:43.957450    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"]
Sep 11 12:04:43.960413 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:04:43.960385    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07e8692a_bc60_4a8d_9dfc_44f347ce2145.slice/crio-2b9f0da45b8655a5d1b0457d4e2e19b58c2f1ee843bd34eff1abb9e5ab137ece WatchSource:0}: Error finding container 2b9f0da45b8655a5d1b0457d4e2e19b58c2f1ee843bd34eff1abb9e5ab137ece: Status 404 returned error can't find the container with id 2b9f0da45b8655a5d1b0457d4e2e19b58c2f1ee843bd34eff1abb9e5ab137ece
Sep 11 12:04:44.798353 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:44.798294    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255" event={"ID":"07e8692a-bc60-4a8d-9dfc-44f347ce2145","Type":"ContainerStarted","Data":"40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda"}
Sep 11 12:04:44.798755 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:44.798396    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255" event={"ID":"07e8692a-bc60-4a8d-9dfc-44f347ce2145","Type":"ContainerStarted","Data":"2b9f0da45b8655a5d1b0457d4e2e19b58c2f1ee843bd34eff1abb9e5ab137ece"}
Sep 11 12:04:44.798755 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:44.798545    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:04:44.800094 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:44.800066    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp" event={"ID":"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b","Type":"ContainerStarted","Data":"8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8"}
Sep 11 12:04:44.800218 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:44.800162    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:44.815522 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:44.815473    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255" podStartSLOduration=1.815453306 podStartE2EDuration="1.815453306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:44.815134571 +0000 UTC m=+793.315854842" watchObservedRunningTime="2026-09-11 12:04:44.815453306 +0000 UTC m=+793.316173574"
Sep 11 12:04:44.831689 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:44.831644    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp" podStartSLOduration=2.083134934 podStartE2EDuration="2.831629472s" totalImagesPullingTime="748.494538ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:04:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:04:44.830934683 +0000 UTC m=+793.331654952" watchObservedRunningTime="2026-09-11 12:04:44.831629472 +0000 UTC m=+793.332349740"
Sep 11 12:04:50.811449 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:50.811417    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:04:55.807520 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:04:55.807443    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:05:13.485342 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.485292    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"]
Sep 11 12:05:13.485837 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.485519    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp" podUID="9ce10dd5-4a81-448b-8a9a-06d0a26fb27b" containerName="maas-api" containerID="cri-o://8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8" gracePeriod=30
Sep 11 12:05:13.549172 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.549129    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"]
Sep 11 12:05:13.549377 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.549351    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255" podUID="07e8692a-bc60-4a8d-9dfc-44f347ce2145" containerName="payload-pre-processing" containerID="cri-o://40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda" gracePeriod=30
Sep 11 12:05:13.816257 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.816222    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:05:13.835139 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.835110    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:05:13.954078 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.954048    2562 generic.go:356] "Generic (PLEG): container finished" podID="07e8692a-bc60-4a8d-9dfc-44f347ce2145" containerID="40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda" exitCode=0
Sep 11 12:05:13.954261 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.954103    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"
Sep 11 12:05:13.954261 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.954123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255" event={"ID":"07e8692a-bc60-4a8d-9dfc-44f347ce2145","Type":"ContainerDied","Data":"40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda"}
Sep 11 12:05:13.954261 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.954185    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255" event={"ID":"07e8692a-bc60-4a8d-9dfc-44f347ce2145","Type":"ContainerDied","Data":"2b9f0da45b8655a5d1b0457d4e2e19b58c2f1ee843bd34eff1abb9e5ab137ece"}
Sep 11 12:05:13.954261 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.954219    2562 scope.go:117] "RemoveContainer" containerID="40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda"
Sep 11 12:05:13.955587 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.955564    2562 generic.go:356] "Generic (PLEG): container finished" podID="9ce10dd5-4a81-448b-8a9a-06d0a26fb27b" containerID="8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8" exitCode=0
Sep 11 12:05:13.955693 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.955624    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"
Sep 11 12:05:13.955693 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.955630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp" event={"ID":"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b","Type":"ContainerDied","Data":"8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8"}
Sep 11 12:05:13.955693 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.955671    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp" event={"ID":"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b","Type":"ContainerDied","Data":"62321973c4d23288f983661ebd07c1f6c11ab5dd5fe2583b00696442d81e989d"}
Sep 11 12:05:13.961255 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.961232    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhc7k\" (UniqueName: \"kubernetes.io/projected/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-kube-api-access-jhc7k\") pod \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") "
Sep 11 12:05:13.961388 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.961284    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-openshift-service-ca\") pod \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") "
Sep 11 12:05:13.961457 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.961404    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07e8692a-bc60-4a8d-9dfc-44f347ce2145-plugins-config-volume\") pod \"07e8692a-bc60-4a8d-9dfc-44f347ce2145\" (UID: \"07e8692a-bc60-4a8d-9dfc-44f347ce2145\") "
Sep 11 12:05:13.961457 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.961432    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-metrics-certs\") pod \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") "
Sep 11 12:05:13.961622 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.961466    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x58lk\" (UniqueName: \"kubernetes.io/projected/07e8692a-bc60-4a8d-9dfc-44f347ce2145-kube-api-access-x58lk\") pod \"07e8692a-bc60-4a8d-9dfc-44f347ce2145\" (UID: \"07e8692a-bc60-4a8d-9dfc-44f347ce2145\") "
Sep 11 12:05:13.961622 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.961534    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-maas-api-tls\") pod \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\" (UID: \"9ce10dd5-4a81-448b-8a9a-06d0a26fb27b\") "
Sep 11 12:05:13.961725 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.961652    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ce10dd5-4a81-448b-8a9a-06d0a26fb27b" (UID: "9ce10dd5-4a81-448b-8a9a-06d0a26fb27b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:05:13.961915 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.961857    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:13.962279 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.962229    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07e8692a-bc60-4a8d-9dfc-44f347ce2145-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "07e8692a-bc60-4a8d-9dfc-44f347ce2145" (UID: "07e8692a-bc60-4a8d-9dfc-44f347ce2145"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:05:13.963845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.963813    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07e8692a-bc60-4a8d-9dfc-44f347ce2145-kube-api-access-x58lk" (OuterVolumeSpecName: "kube-api-access-x58lk") pod "07e8692a-bc60-4a8d-9dfc-44f347ce2145" (UID: "07e8692a-bc60-4a8d-9dfc-44f347ce2145"). InnerVolumeSpecName "kube-api-access-x58lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:05:13.963960 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.963936    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ce10dd5-4a81-448b-8a9a-06d0a26fb27b" (UID: "9ce10dd5-4a81-448b-8a9a-06d0a26fb27b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:05:13.964013 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.963986    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9ce10dd5-4a81-448b-8a9a-06d0a26fb27b" (UID: "9ce10dd5-4a81-448b-8a9a-06d0a26fb27b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:05:13.964061 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.964043    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-kube-api-access-jhc7k" (OuterVolumeSpecName: "kube-api-access-jhc7k") pod "9ce10dd5-4a81-448b-8a9a-06d0a26fb27b" (UID: "9ce10dd5-4a81-448b-8a9a-06d0a26fb27b"). InnerVolumeSpecName "kube-api-access-jhc7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:05:13.964769 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.964754    2562 scope.go:117] "RemoveContainer" containerID="40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda"
Sep 11 12:05:13.965050 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:05:13.965032    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda\": container with ID starting with 40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda not found: ID does not exist" containerID="40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda"
Sep 11 12:05:13.965094 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.965057    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda"} err="failed to get container status \"40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda\": rpc error: code = NotFound desc = could not find container \"40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda\": container with ID starting with 40d56897259ade6132e35fbaf53e24ec5a78a0f9f758318a1d7ab63fd1fb3fda not found: ID does not exist"
Sep 11 12:05:13.965094 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.965072    2562 scope.go:117] "RemoveContainer" containerID="8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8"
Sep 11 12:05:13.976281 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.976262    2562 scope.go:117] "RemoveContainer" containerID="8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8"
Sep 11 12:05:13.976562 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:05:13.976542    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8\": container with ID starting with 8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8 not found: ID does not exist" containerID="8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8"
Sep 11 12:05:13.976614 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:13.976569    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8"} err="failed to get container status \"8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8\": rpc error: code = NotFound desc = could not find container \"8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8\": container with ID starting with 8174d263e1ab262540ab7870fe8021138803ebca6ce577fa5d2bd513ecf9d5f8 not found: ID does not exist"
Sep 11 12:05:14.063428 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:14.063396    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhc7k\" (UniqueName: \"kubernetes.io/projected/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-kube-api-access-jhc7k\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:14.063585 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:14.063438    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07e8692a-bc60-4a8d-9dfc-44f347ce2145-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:14.063585 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:14.063456    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:14.063585 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:14.063469    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x58lk\" (UniqueName: \"kubernetes.io/projected/07e8692a-bc60-4a8d-9dfc-44f347ce2145-kube-api-access-x58lk\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:14.063585 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:14.063477    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:14.271880 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:14.271844    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"]
Sep 11 12:05:14.279338 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:14.278690    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6d3d15b-668b958986-m8wdp"]
Sep 11 12:05:14.291857 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:14.291782    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"]
Sep 11 12:05:14.296151 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:14.296121    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6d3d15b-68558d7c4b-27255"]
Sep 11 12:05:16.024616 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:16.024585    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07e8692a-bc60-4a8d-9dfc-44f347ce2145" path="/var/lib/kubelet/pods/07e8692a-bc60-4a8d-9dfc-44f347ce2145/volumes"
Sep 11 12:05:16.024987 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:16.024969    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce10dd5-4a81-448b-8a9a-06d0a26fb27b" path="/var/lib/kubelet/pods/9ce10dd5-4a81-448b-8a9a-06d0a26fb27b/volumes"
Sep 11 12:05:18.843639 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:18.843601    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"]
Sep 11 12:05:18.844029 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:18.844014    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce10dd5-4a81-448b-8a9a-06d0a26fb27b" containerName="maas-api"
Sep 11 12:05:18.844029 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:18.844029    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce10dd5-4a81-448b-8a9a-06d0a26fb27b" containerName="maas-api"
Sep 11 12:05:18.844122 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:18.844039    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07e8692a-bc60-4a8d-9dfc-44f347ce2145" containerName="payload-pre-processing"
Sep 11 12:05:18.844122 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:18.844044    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="07e8692a-bc60-4a8d-9dfc-44f347ce2145" containerName="payload-pre-processing"
Sep 11 12:05:18.844203 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:18.844131    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce10dd5-4a81-448b-8a9a-06d0a26fb27b" containerName="maas-api"
Sep 11 12:05:18.844203 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:18.844145    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="07e8692a-bc60-4a8d-9dfc-44f347ce2145" containerName="payload-pre-processing"
Sep 11 12:05:18.848990 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:18.848966    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:18.851511 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:18.851483    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-24d087d2-serving-cert\""
Sep 11 12:05:18.860292 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:18.860200    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"]
Sep 11 12:05:19.013132 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.013102    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2flkp\" (UniqueName: \"kubernetes.io/projected/7868f05c-0fce-47b1-9b15-476f7a295869-kube-api-access-2flkp\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.013326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.013143    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7868f05c-0fce-47b1-9b15-476f7a295869-openshift-service-ca\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.013326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.013168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-metrics-certs\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.013326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.013255    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-maas-api-tls\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.114826 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.114731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2flkp\" (UniqueName: \"kubernetes.io/projected/7868f05c-0fce-47b1-9b15-476f7a295869-kube-api-access-2flkp\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.114826 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.114799    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7868f05c-0fce-47b1-9b15-476f7a295869-openshift-service-ca\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.114826 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.114827    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-metrics-certs\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.115104 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.114878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-maas-api-tls\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.115577 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.115552    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7868f05c-0fce-47b1-9b15-476f7a295869-openshift-service-ca\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.117350 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.117320    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-metrics-certs\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.117485 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.117463    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-maas-api-tls\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.122649 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.122625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2flkp\" (UniqueName: \"kubernetes.io/projected/7868f05c-0fce-47b1-9b15-476f7a295869-kube-api-access-2flkp\") pod \"maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.161986 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.161955    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:19.460922 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.460802    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"]
Sep 11 12:05:19.463727 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:05:19.463699    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7868f05c_0fce_47b1_9b15_476f7a295869.slice/crio-b0ddc42adfa1a3bb6ec4e3e85be953e6bd5afbf89d25e54048200b9efc130745 WatchSource:0}: Error finding container b0ddc42adfa1a3bb6ec4e3e85be953e6bd5afbf89d25e54048200b9efc130745: Status 404 returned error can't find the container with id b0ddc42adfa1a3bb6ec4e3e85be953e6bd5afbf89d25e54048200b9efc130745
Sep 11 12:05:19.611980 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.611945    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"]
Sep 11 12:05:19.616891 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.616865    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:19.619916 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.619891    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-24d087d2\""
Sep 11 12:05:19.637252 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.637222    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"]
Sep 11 12:05:19.721128 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.720987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e5fbe15-1e98-483c-8efb-5660a3d26c74-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc\" (UID: \"5e5fbe15-1e98-483c-8efb-5660a3d26c74\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:19.721421 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.721394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddfxd\" (UniqueName: \"kubernetes.io/projected/5e5fbe15-1e98-483c-8efb-5660a3d26c74-kube-api-access-ddfxd\") pod \"payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc\" (UID: \"5e5fbe15-1e98-483c-8efb-5660a3d26c74\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:19.822822 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.822781    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e5fbe15-1e98-483c-8efb-5660a3d26c74-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc\" (UID: \"5e5fbe15-1e98-483c-8efb-5660a3d26c74\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:19.822997 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.822830    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddfxd\" (UniqueName: \"kubernetes.io/projected/5e5fbe15-1e98-483c-8efb-5660a3d26c74-kube-api-access-ddfxd\") pod \"payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc\" (UID: \"5e5fbe15-1e98-483c-8efb-5660a3d26c74\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:19.823633 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.823612    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e5fbe15-1e98-483c-8efb-5660a3d26c74-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc\" (UID: \"5e5fbe15-1e98-483c-8efb-5660a3d26c74\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:19.831037 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.831010    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddfxd\" (UniqueName: \"kubernetes.io/projected/5e5fbe15-1e98-483c-8efb-5660a3d26c74-kube-api-access-ddfxd\") pod \"payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc\" (UID: \"5e5fbe15-1e98-483c-8efb-5660a3d26c74\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:19.927723 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.927700    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:19.995341 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.995294    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6" event={"ID":"7868f05c-0fce-47b1-9b15-476f7a295869","Type":"ContainerStarted","Data":"47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52"}
Sep 11 12:05:19.995461 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:19.995359    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6" event={"ID":"7868f05c-0fce-47b1-9b15-476f7a295869","Type":"ContainerStarted","Data":"b0ddc42adfa1a3bb6ec4e3e85be953e6bd5afbf89d25e54048200b9efc130745"}
Sep 11 12:05:20.214479 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:20.214448    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"]
Sep 11 12:05:21.002058 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:21.002015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc" event={"ID":"5e5fbe15-1e98-483c-8efb-5660a3d26c74","Type":"ContainerStarted","Data":"dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c"}
Sep 11 12:05:21.002542 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:21.002072    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc" event={"ID":"5e5fbe15-1e98-483c-8efb-5660a3d26c74","Type":"ContainerStarted","Data":"5a54c9e600a90ed583c788d158c8882e827919bf9daf05480799e144d86e6958"}
Sep 11 12:05:21.002542 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:21.002102    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:21.002542 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:21.002257    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:21.017020 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:21.016963    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6" podStartSLOduration=2.566167582 podStartE2EDuration="3.01694488s" totalImagesPullingTime="450.777298ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:05:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:05:21.016080217 +0000 UTC m=+829.516800486" watchObservedRunningTime="2026-09-11 12:05:21.01694488 +0000 UTC m=+829.517665149"
Sep 11 12:05:21.032035 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:21.031991    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc" podStartSLOduration=2.031975281 podStartE2EDuration="2.031975281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:05:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:05:21.030055549 +0000 UTC m=+829.530775816" watchObservedRunningTime="2026-09-11 12:05:21.031975281 +0000 UTC m=+829.532695549"
Sep 11 12:05:27.013455 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:27.013422    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:32.010352 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:32.010314    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:49.676609 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:49.676559    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"]
Sep 11 12:05:49.677074 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:49.676819    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6" podUID="7868f05c-0fce-47b1-9b15-476f7a295869" containerName="maas-api" containerID="cri-o://47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52" gracePeriod=30
Sep 11 12:05:49.754626 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:49.754588    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"]
Sep 11 12:05:49.755000 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:49.754847    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc" podUID="5e5fbe15-1e98-483c-8efb-5660a3d26c74" containerName="payload-pre-processing" containerID="cri-o://dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c" gracePeriod=30
Sep 11 12:05:50.013731 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.013702    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:50.066786 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.066762    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:50.107250 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.107211    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-metrics-certs\") pod \"7868f05c-0fce-47b1-9b15-476f7a295869\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") "
Sep 11 12:05:50.107250 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.107256    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-maas-api-tls\") pod \"7868f05c-0fce-47b1-9b15-476f7a295869\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") "
Sep 11 12:05:50.107490 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.107368    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2flkp\" (UniqueName: \"kubernetes.io/projected/7868f05c-0fce-47b1-9b15-476f7a295869-kube-api-access-2flkp\") pod \"7868f05c-0fce-47b1-9b15-476f7a295869\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") "
Sep 11 12:05:50.107490 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.107397    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7868f05c-0fce-47b1-9b15-476f7a295869-openshift-service-ca\") pod \"7868f05c-0fce-47b1-9b15-476f7a295869\" (UID: \"7868f05c-0fce-47b1-9b15-476f7a295869\") "
Sep 11 12:05:50.107833 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.107801    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7868f05c-0fce-47b1-9b15-476f7a295869-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7868f05c-0fce-47b1-9b15-476f7a295869" (UID: "7868f05c-0fce-47b1-9b15-476f7a295869"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:05:50.109463 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.109440    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7868f05c-0fce-47b1-9b15-476f7a295869" (UID: "7868f05c-0fce-47b1-9b15-476f7a295869"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:05:50.109874 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.109846    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7868f05c-0fce-47b1-9b15-476f7a295869" (UID: "7868f05c-0fce-47b1-9b15-476f7a295869"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:05:50.109937 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.109849    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7868f05c-0fce-47b1-9b15-476f7a295869-kube-api-access-2flkp" (OuterVolumeSpecName: "kube-api-access-2flkp") pod "7868f05c-0fce-47b1-9b15-476f7a295869" (UID: "7868f05c-0fce-47b1-9b15-476f7a295869"). InnerVolumeSpecName "kube-api-access-2flkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:05:50.145410 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.145374    2562 generic.go:356] "Generic (PLEG): container finished" podID="5e5fbe15-1e98-483c-8efb-5660a3d26c74" containerID="dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c" exitCode=0
Sep 11 12:05:50.145601 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.145423    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"
Sep 11 12:05:50.145601 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.145474    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc" event={"ID":"5e5fbe15-1e98-483c-8efb-5660a3d26c74","Type":"ContainerDied","Data":"dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c"}
Sep 11 12:05:50.145601 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.145545    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc" event={"ID":"5e5fbe15-1e98-483c-8efb-5660a3d26c74","Type":"ContainerDied","Data":"5a54c9e600a90ed583c788d158c8882e827919bf9daf05480799e144d86e6958"}
Sep 11 12:05:50.145601 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.145578    2562 scope.go:117] "RemoveContainer" containerID="dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c"
Sep 11 12:05:50.147025 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.147004    2562 generic.go:356] "Generic (PLEG): container finished" podID="7868f05c-0fce-47b1-9b15-476f7a295869" containerID="47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52" exitCode=0
Sep 11 12:05:50.147106 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.147041    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"
Sep 11 12:05:50.147106 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.147044    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6" event={"ID":"7868f05c-0fce-47b1-9b15-476f7a295869","Type":"ContainerDied","Data":"47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52"}
Sep 11 12:05:50.147106 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.147094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6" event={"ID":"7868f05c-0fce-47b1-9b15-476f7a295869","Type":"ContainerDied","Data":"b0ddc42adfa1a3bb6ec4e3e85be953e6bd5afbf89d25e54048200b9efc130745"}
Sep 11 12:05:50.155231 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.155209    2562 scope.go:117] "RemoveContainer" containerID="dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c"
Sep 11 12:05:50.155617 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:05:50.155542    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c\": container with ID starting with dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c not found: ID does not exist" containerID="dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c"
Sep 11 12:05:50.155617 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.155574    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c"} err="failed to get container status \"dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c\": rpc error: code = NotFound desc = could not find container \"dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c\": container with ID starting with dfe77425760f0f33d4d4987d22af01378a7ab4dbc2152e0eb88b1bd01d5c8b4c not found: ID does not exist"
Sep 11 12:05:50.155617 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.155597    2562 scope.go:117] "RemoveContainer" containerID="47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52"
Sep 11 12:05:50.164001 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.163984    2562 scope.go:117] "RemoveContainer" containerID="47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52"
Sep 11 12:05:50.164263 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:05:50.164242    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52\": container with ID starting with 47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52 not found: ID does not exist" containerID="47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52"
Sep 11 12:05:50.164313 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.164271    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52"} err="failed to get container status \"47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52\": rpc error: code = NotFound desc = could not find container \"47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52\": container with ID starting with 47517ca6e0dbbcf8146f8e6b87decf61e814af9488b89838583dd3cb7a8dda52 not found: ID does not exist"
Sep 11 12:05:50.168026 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.168004    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"]
Sep 11 12:05:50.173997 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.173973    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24d087d2-787bcdc868-xvzg6"]
Sep 11 12:05:50.207875 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.207842    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e5fbe15-1e98-483c-8efb-5660a3d26c74-plugins-config-volume\") pod \"5e5fbe15-1e98-483c-8efb-5660a3d26c74\" (UID: \"5e5fbe15-1e98-483c-8efb-5660a3d26c74\") "
Sep 11 12:05:50.207875 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.207882    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddfxd\" (UniqueName: \"kubernetes.io/projected/5e5fbe15-1e98-483c-8efb-5660a3d26c74-kube-api-access-ddfxd\") pod \"5e5fbe15-1e98-483c-8efb-5660a3d26c74\" (UID: \"5e5fbe15-1e98-483c-8efb-5660a3d26c74\") "
Sep 11 12:05:50.208165 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.208151    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2flkp\" (UniqueName: \"kubernetes.io/projected/7868f05c-0fce-47b1-9b15-476f7a295869-kube-api-access-2flkp\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:50.208165 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.208165    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7868f05c-0fce-47b1-9b15-476f7a295869-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:50.208287 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.208173    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:50.208287 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.208184    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7868f05c-0fce-47b1-9b15-476f7a295869-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:50.208287 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.208195    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e5fbe15-1e98-483c-8efb-5660a3d26c74-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e5fbe15-1e98-483c-8efb-5660a3d26c74" (UID: "5e5fbe15-1e98-483c-8efb-5660a3d26c74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:05:50.209991 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.209968    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e5fbe15-1e98-483c-8efb-5660a3d26c74-kube-api-access-ddfxd" (OuterVolumeSpecName: "kube-api-access-ddfxd") pod "5e5fbe15-1e98-483c-8efb-5660a3d26c74" (UID: "5e5fbe15-1e98-483c-8efb-5660a3d26c74"). InnerVolumeSpecName "kube-api-access-ddfxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:05:50.309766 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.309674    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e5fbe15-1e98-483c-8efb-5660a3d26c74-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:50.309766 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.309710    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddfxd\" (UniqueName: \"kubernetes.io/projected/5e5fbe15-1e98-483c-8efb-5660a3d26c74-kube-api-access-ddfxd\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:05:50.467591 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.467557    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"]
Sep 11 12:05:50.474330 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:50.470784    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-24d087d2-5fb846d7f4-pnmvc"]
Sep 11 12:05:52.025111 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:52.025073    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e5fbe15-1e98-483c-8efb-5660a3d26c74" path="/var/lib/kubelet/pods/5e5fbe15-1e98-483c-8efb-5660a3d26c74/volumes"
Sep 11 12:05:52.025507 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:52.025476    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7868f05c-0fce-47b1-9b15-476f7a295869" path="/var/lib/kubelet/pods/7868f05c-0fce-47b1-9b15-476f7a295869/volumes"
Sep 11 12:05:55.923663 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:55.923623    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"]
Sep 11 12:05:55.924150 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:55.924038    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e5fbe15-1e98-483c-8efb-5660a3d26c74" containerName="payload-pre-processing"
Sep 11 12:05:55.924150 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:55.924050    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e5fbe15-1e98-483c-8efb-5660a3d26c74" containerName="payload-pre-processing"
Sep 11 12:05:55.924150 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:55.924059    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7868f05c-0fce-47b1-9b15-476f7a295869" containerName="maas-api"
Sep 11 12:05:55.924150 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:55.924065    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7868f05c-0fce-47b1-9b15-476f7a295869" containerName="maas-api"
Sep 11 12:05:55.924150 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:55.924138    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e5fbe15-1e98-483c-8efb-5660a3d26c74" containerName="payload-pre-processing"
Sep 11 12:05:55.924150 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:55.924148    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7868f05c-0fce-47b1-9b15-476f7a295869" containerName="maas-api"
Sep 11 12:05:55.928575 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:55.928555    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:55.930861 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:55.930839    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-5e4884c8-serving-cert\""
Sep 11 12:05:55.938128 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:55.937433    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"]
Sep 11 12:05:56.064747 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.064710    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a57b92bd-ce1f-4755-abcf-e6348ba75231-openshift-service-ca\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.064946 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.064770    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-metrics-certs\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.064946 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.064897    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-maas-api-tls\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.065059 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.064994    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dssp\" (UniqueName: \"kubernetes.io/projected/a57b92bd-ce1f-4755-abcf-e6348ba75231-kube-api-access-4dssp\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.166439 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.166398    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-maas-api-tls\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.166631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.166469    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dssp\" (UniqueName: \"kubernetes.io/projected/a57b92bd-ce1f-4755-abcf-e6348ba75231-kube-api-access-4dssp\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.166631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.166521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a57b92bd-ce1f-4755-abcf-e6348ba75231-openshift-service-ca\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.166631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.166566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-metrics-certs\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.167248 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.167222    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a57b92bd-ce1f-4755-abcf-e6348ba75231-openshift-service-ca\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.168901 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.168881    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-maas-api-tls\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.169010 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.168994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-metrics-certs\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.177023 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.176956    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dssp\" (UniqueName: \"kubernetes.io/projected/a57b92bd-ce1f-4755-abcf-e6348ba75231-kube-api-access-4dssp\") pod \"maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.242502 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.242471    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:56.516031 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.516003    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"]
Sep 11 12:05:56.518328 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:05:56.518275    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57b92bd_ce1f_4755_abcf_e6348ba75231.slice/crio-ac95557dabf9fc572bafafb80f737feb399c4802217052b0001a7d308b9f0b8f WatchSource:0}: Error finding container ac95557dabf9fc572bafafb80f737feb399c4802217052b0001a7d308b9f0b8f: Status 404 returned error can't find the container with id ac95557dabf9fc572bafafb80f737feb399c4802217052b0001a7d308b9f0b8f
Sep 11 12:05:56.573076 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.573043    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"]
Sep 11 12:05:56.579809 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.579774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:05:56.583467 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.583431    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-5e4884c8\""
Sep 11 12:05:56.583619 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.583536    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-5e4884c8-dockercfg-s7b8l\""
Sep 11 12:05:56.594657 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.594627    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"]
Sep 11 12:05:56.672148 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.672104    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn2bc\" (UniqueName: \"kubernetes.io/projected/47969525-db83-45cb-95a9-9a5e99d8faf7-kube-api-access-mn2bc\") pod \"payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr\" (UID: \"47969525-db83-45cb-95a9-9a5e99d8faf7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:05:56.672553 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.672520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47969525-db83-45cb-95a9-9a5e99d8faf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr\" (UID: \"47969525-db83-45cb-95a9-9a5e99d8faf7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:05:56.773796 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.773692    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn2bc\" (UniqueName: \"kubernetes.io/projected/47969525-db83-45cb-95a9-9a5e99d8faf7-kube-api-access-mn2bc\") pod \"payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr\" (UID: \"47969525-db83-45cb-95a9-9a5e99d8faf7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:05:56.773796 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.773770    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47969525-db83-45cb-95a9-9a5e99d8faf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr\" (UID: \"47969525-db83-45cb-95a9-9a5e99d8faf7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:05:56.774530 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.774505    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47969525-db83-45cb-95a9-9a5e99d8faf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr\" (UID: \"47969525-db83-45cb-95a9-9a5e99d8faf7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:05:56.794768 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.794734    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn2bc\" (UniqueName: \"kubernetes.io/projected/47969525-db83-45cb-95a9-9a5e99d8faf7-kube-api-access-mn2bc\") pod \"payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr\" (UID: \"47969525-db83-45cb-95a9-9a5e99d8faf7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:05:56.891571 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:56.891544    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:05:57.171105 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:57.171061    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"]
Sep 11 12:05:57.173557 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:05:57.173532    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47969525_db83_45cb_95a9_9a5e99d8faf7.slice/crio-2ad55ffbbaeb25e20b8b38aa9da2f76c6b9df3b20fb7e8ba0d3cb52ce90d236b WatchSource:0}: Error finding container 2ad55ffbbaeb25e20b8b38aa9da2f76c6b9df3b20fb7e8ba0d3cb52ce90d236b: Status 404 returned error can't find the container with id 2ad55ffbbaeb25e20b8b38aa9da2f76c6b9df3b20fb7e8ba0d3cb52ce90d236b
Sep 11 12:05:57.188754 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:57.188723    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2" event={"ID":"a57b92bd-ce1f-4755-abcf-e6348ba75231","Type":"ContainerStarted","Data":"723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759"}
Sep 11 12:05:57.189528 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:57.188789    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2" event={"ID":"a57b92bd-ce1f-4755-abcf-e6348ba75231","Type":"ContainerStarted","Data":"ac95557dabf9fc572bafafb80f737feb399c4802217052b0001a7d308b9f0b8f"}
Sep 11 12:05:57.189528 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:57.188849    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:05:57.190498 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:57.190473    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr" event={"ID":"47969525-db83-45cb-95a9-9a5e99d8faf7","Type":"ContainerStarted","Data":"2ad55ffbbaeb25e20b8b38aa9da2f76c6b9df3b20fb7e8ba0d3cb52ce90d236b"}
Sep 11 12:05:57.206466 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:57.206422    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2" podStartSLOduration=1.8430276349999999 podStartE2EDuration="2.206405292s" totalImagesPullingTime="363.377657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:05:57.205323567 +0000 UTC m=+865.706043832" watchObservedRunningTime="2026-09-11 12:05:57.206405292 +0000 UTC m=+865.707125627"
Sep 11 12:05:58.198448 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:58.198407    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr" event={"ID":"47969525-db83-45cb-95a9-9a5e99d8faf7","Type":"ContainerStarted","Data":"4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6"}
Sep 11 12:05:58.216153 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:58.216091    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr" podStartSLOduration=2.216071876 podStartE2EDuration="2.216071876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:05:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:05:58.214548323 +0000 UTC m=+866.715268588" watchObservedRunningTime="2026-09-11 12:05:58.216071876 +0000 UTC m=+866.716792203"
Sep 11 12:05:59.204298 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:05:59.204264    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:06:03.204043 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:03.204008    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:06:10.211419 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:10.211381    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:06:26.463396 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.463314    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"]
Sep 11 12:06:26.463836 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.463551    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2" podUID="a57b92bd-ce1f-4755-abcf-e6348ba75231" containerName="maas-api" containerID="cri-o://723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759" gracePeriod=30
Sep 11 12:06:26.517984 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.517946    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"]
Sep 11 12:06:26.518435 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.518197    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr" podUID="47969525-db83-45cb-95a9-9a5e99d8faf7" containerName="payload-pre-processing" containerID="cri-o://4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6" gracePeriod=30
Sep 11 12:06:26.772299 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.772271    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:06:26.816809 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.816778    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:06:26.862871 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.862840    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47969525-db83-45cb-95a9-9a5e99d8faf7-plugins-config-volume\") pod \"47969525-db83-45cb-95a9-9a5e99d8faf7\" (UID: \"47969525-db83-45cb-95a9-9a5e99d8faf7\") "
Sep 11 12:06:26.863071 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.862918    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-metrics-certs\") pod \"a57b92bd-ce1f-4755-abcf-e6348ba75231\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") "
Sep 11 12:06:26.863071 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.862962    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dssp\" (UniqueName: \"kubernetes.io/projected/a57b92bd-ce1f-4755-abcf-e6348ba75231-kube-api-access-4dssp\") pod \"a57b92bd-ce1f-4755-abcf-e6348ba75231\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") "
Sep 11 12:06:26.863071 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.863023    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-maas-api-tls\") pod \"a57b92bd-ce1f-4755-abcf-e6348ba75231\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") "
Sep 11 12:06:26.863071 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.863056    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn2bc\" (UniqueName: \"kubernetes.io/projected/47969525-db83-45cb-95a9-9a5e99d8faf7-kube-api-access-mn2bc\") pod \"47969525-db83-45cb-95a9-9a5e99d8faf7\" (UID: \"47969525-db83-45cb-95a9-9a5e99d8faf7\") "
Sep 11 12:06:26.863282 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.863105    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a57b92bd-ce1f-4755-abcf-e6348ba75231-openshift-service-ca\") pod \"a57b92bd-ce1f-4755-abcf-e6348ba75231\" (UID: \"a57b92bd-ce1f-4755-abcf-e6348ba75231\") "
Sep 11 12:06:26.863350 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.863321    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47969525-db83-45cb-95a9-9a5e99d8faf7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "47969525-db83-45cb-95a9-9a5e99d8faf7" (UID: "47969525-db83-45cb-95a9-9a5e99d8faf7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:06:26.863566 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.863539    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a57b92bd-ce1f-4755-abcf-e6348ba75231-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a57b92bd-ce1f-4755-abcf-e6348ba75231" (UID: "a57b92bd-ce1f-4755-abcf-e6348ba75231"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:06:26.863654 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.863615    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47969525-db83-45cb-95a9-9a5e99d8faf7-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:26.865291 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.865254    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a57b92bd-ce1f-4755-abcf-e6348ba75231" (UID: "a57b92bd-ce1f-4755-abcf-e6348ba75231"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:06:26.865421 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.865290    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47969525-db83-45cb-95a9-9a5e99d8faf7-kube-api-access-mn2bc" (OuterVolumeSpecName: "kube-api-access-mn2bc") pod "47969525-db83-45cb-95a9-9a5e99d8faf7" (UID: "47969525-db83-45cb-95a9-9a5e99d8faf7"). InnerVolumeSpecName "kube-api-access-mn2bc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:26.865555 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.865536    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a57b92bd-ce1f-4755-abcf-e6348ba75231-kube-api-access-4dssp" (OuterVolumeSpecName: "kube-api-access-4dssp") pod "a57b92bd-ce1f-4755-abcf-e6348ba75231" (UID: "a57b92bd-ce1f-4755-abcf-e6348ba75231"). InnerVolumeSpecName "kube-api-access-4dssp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:26.865606 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.865535    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a57b92bd-ce1f-4755-abcf-e6348ba75231" (UID: "a57b92bd-ce1f-4755-abcf-e6348ba75231"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:06:26.964362 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.964278    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:26.964362 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.964354    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn2bc\" (UniqueName: \"kubernetes.io/projected/47969525-db83-45cb-95a9-9a5e99d8faf7-kube-api-access-mn2bc\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:26.964362 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.964369    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a57b92bd-ce1f-4755-abcf-e6348ba75231-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:26.964630 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.964381    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a57b92bd-ce1f-4755-abcf-e6348ba75231-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:26.964630 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:26.964393    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dssp\" (UniqueName: \"kubernetes.io/projected/a57b92bd-ce1f-4755-abcf-e6348ba75231-kube-api-access-4dssp\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:27.351005 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.350965    2562 generic.go:356] "Generic (PLEG): container finished" podID="a57b92bd-ce1f-4755-abcf-e6348ba75231" containerID="723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759" exitCode=0
Sep 11 12:06:27.351173 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.351027    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"
Sep 11 12:06:27.351173 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.351055    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2" event={"ID":"a57b92bd-ce1f-4755-abcf-e6348ba75231","Type":"ContainerDied","Data":"723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759"}
Sep 11 12:06:27.351173 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.351108    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2" event={"ID":"a57b92bd-ce1f-4755-abcf-e6348ba75231","Type":"ContainerDied","Data":"ac95557dabf9fc572bafafb80f737feb399c4802217052b0001a7d308b9f0b8f"}
Sep 11 12:06:27.351173 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.351138    2562 scope.go:117] "RemoveContainer" containerID="723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759"
Sep 11 12:06:27.352550 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.352525    2562 generic.go:356] "Generic (PLEG): container finished" podID="47969525-db83-45cb-95a9-9a5e99d8faf7" containerID="4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6" exitCode=0
Sep 11 12:06:27.352687 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.352581    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"
Sep 11 12:06:27.352687 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.352633    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr" event={"ID":"47969525-db83-45cb-95a9-9a5e99d8faf7","Type":"ContainerDied","Data":"4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6"}
Sep 11 12:06:27.352687 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.352673    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr" event={"ID":"47969525-db83-45cb-95a9-9a5e99d8faf7","Type":"ContainerDied","Data":"2ad55ffbbaeb25e20b8b38aa9da2f76c6b9df3b20fb7e8ba0d3cb52ce90d236b"}
Sep 11 12:06:27.361780 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.361758    2562 scope.go:117] "RemoveContainer" containerID="723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759"
Sep 11 12:06:27.362070 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:06:27.362051    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759\": container with ID starting with 723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759 not found: ID does not exist" containerID="723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759"
Sep 11 12:06:27.362152 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.362082    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759"} err="failed to get container status \"723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759\": rpc error: code = NotFound desc = could not find container \"723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759\": container with ID starting with 723dcca268a09de8efb3043ec39b21103474037449166f0423b9a9606d7b7759 not found: ID does not exist"
Sep 11 12:06:27.362152 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.362106    2562 scope.go:117] "RemoveContainer" containerID="4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6"
Sep 11 12:06:27.371435 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.371415    2562 scope.go:117] "RemoveContainer" containerID="4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6"
Sep 11 12:06:27.371708 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:06:27.371690    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6\": container with ID starting with 4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6 not found: ID does not exist" containerID="4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6"
Sep 11 12:06:27.371754 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.371718    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6"} err="failed to get container status \"4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6\": rpc error: code = NotFound desc = could not find container \"4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6\": container with ID starting with 4811db24321edfa03e083e4dc811b67fe457da77d2ceb3f1ee9314dd46dd13a6 not found: ID does not exist"
Sep 11 12:06:27.377184 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.377162    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"]
Sep 11 12:06:27.380614 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.380591    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5e4884c8-5445465c98-t5psr"]
Sep 11 12:06:27.389751 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.389732    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"]
Sep 11 12:06:27.393039 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:27.393018    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5e4884c8-66bbbffb97-4ccb2"]
Sep 11 12:06:28.026691 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:28.026659    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47969525-db83-45cb-95a9-9a5e99d8faf7" path="/var/lib/kubelet/pods/47969525-db83-45cb-95a9-9a5e99d8faf7/volumes"
Sep 11 12:06:28.027065 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:28.027048    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a57b92bd-ce1f-4755-abcf-e6348ba75231" path="/var/lib/kubelet/pods/a57b92bd-ce1f-4755-abcf-e6348ba75231/volumes"
Sep 11 12:06:40.602425 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.602386    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"]
Sep 11 12:06:40.604917 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.602842    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47969525-db83-45cb-95a9-9a5e99d8faf7" containerName="payload-pre-processing"
Sep 11 12:06:40.604917 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.602856    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="47969525-db83-45cb-95a9-9a5e99d8faf7" containerName="payload-pre-processing"
Sep 11 12:06:40.604917 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.602865    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a57b92bd-ce1f-4755-abcf-e6348ba75231" containerName="maas-api"
Sep 11 12:06:40.604917 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.602871    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a57b92bd-ce1f-4755-abcf-e6348ba75231" containerName="maas-api"
Sep 11 12:06:40.604917 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.602949    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a57b92bd-ce1f-4755-abcf-e6348ba75231" containerName="maas-api"
Sep 11 12:06:40.604917 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.602960    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="47969525-db83-45cb-95a9-9a5e99d8faf7" containerName="payload-pre-processing"
Sep 11 12:06:40.605929 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.605913    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.609052 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.608996    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-34511f\"/\"default-dockercfg-r9qg8\""
Sep 11 12:06:40.609699 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.609681    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-34511f\"/\"kube-root-ca.crt\""
Sep 11 12:06:40.609997 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.609967    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-34511f\"/\"openshift-service-ca.crt\""
Sep 11 12:06:40.610088 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.610028    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-34511f\"/\"test-model-34511f-kserve-self-signed-certs\""
Sep 11 12:06:40.616971 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.616947    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"]
Sep 11 12:06:40.702536 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.702502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-tmp-dir\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.702536 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.702535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9dbca512-ea16-4154-8a79-922a4780be51-tls-certs\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.702753 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.702558    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-dshm\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.702753 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.702630    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzq6t\" (UniqueName: \"kubernetes.io/projected/9dbca512-ea16-4154-8a79-922a4780be51-kube-api-access-jzq6t\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.702753 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.702680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-home\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.702857 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.702758    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-model-cache\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.803975 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.803934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-tmp-dir\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.803975 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.803975    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9dbca512-ea16-4154-8a79-922a4780be51-tls-certs\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.804187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.804005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-dshm\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.804187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.804043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzq6t\" (UniqueName: \"kubernetes.io/projected/9dbca512-ea16-4154-8a79-922a4780be51-kube-api-access-jzq6t\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.804187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.804066    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-home\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.804187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.804143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-model-cache\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.804464 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.804444    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-tmp-dir\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.804548 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.804524    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-home\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.804604 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.804545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-model-cache\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.806454 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.806432    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-dshm\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.806624 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.806609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9dbca512-ea16-4154-8a79-922a4780be51-tls-certs\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.813079 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.813051    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzq6t\" (UniqueName: \"kubernetes.io/projected/9dbca512-ea16-4154-8a79-922a4780be51-kube-api-access-jzq6t\") pod \"test-model-34511f-kserve-7458d56f4f-fsssp\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") " pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:40.916472 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:40.916373    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:41.190716 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:41.190691    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"]
Sep 11 12:06:41.192871 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:06:41.192839    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9dbca512_ea16_4154_8a79_922a4780be51.slice/crio-c5f4fe9d0db90d50a8265ce64f0a8cb1a1824422e85f8c95b2e294dfbfabcf05 WatchSource:0}: Error finding container c5f4fe9d0db90d50a8265ce64f0a8cb1a1824422e85f8c95b2e294dfbfabcf05: Status 404 returned error can't find the container with id c5f4fe9d0db90d50a8265ce64f0a8cb1a1824422e85f8c95b2e294dfbfabcf05
Sep 11 12:06:41.433256 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:41.433217    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp" event={"ID":"9dbca512-ea16-4154-8a79-922a4780be51","Type":"ContainerStarted","Data":"c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66"}
Sep 11 12:06:41.433453 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:41.433265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp" event={"ID":"9dbca512-ea16-4154-8a79-922a4780be51","Type":"ContainerStarted","Data":"c5f4fe9d0db90d50a8265ce64f0a8cb1a1824422e85f8c95b2e294dfbfabcf05"}
Sep 11 12:06:41.451920 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:41.451805    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp" podStartSLOduration=1.451785946 podStartE2EDuration="1.451785946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:06:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:06:41.449991119 +0000 UTC m=+909.950711388" watchObservedRunningTime="2026-09-11 12:06:41.451785946 +0000 UTC m=+909.952506218"
Sep 11 12:06:50.916556 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:50.916506    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:50.917034 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:50.916577    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:50.929244 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:50.929221    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:51.492983 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:51.492949    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:06:54.707606 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:54.707565    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"]
Sep 11 12:06:54.708133 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:54.707856    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb" podUID="eb6304ea-12a8-484b-be5d-c074e1020907" containerName="maas-api" containerID="cri-o://90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82" gracePeriod=30
Sep 11 12:06:54.771516 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:54.771481    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"]
Sep 11 12:06:54.771773 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:54.771746    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn" podUID="b752d36b-1b53-4f5c-9995-dbbc28e7c171" containerName="payload-pre-processing" containerID="cri-o://f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3" gracePeriod=30
Sep 11 12:06:55.022832 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.022803    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:06:55.045225 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.045191    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb6304ea-12a8-484b-be5d-c074e1020907-openshift-service-ca\") pod \"eb6304ea-12a8-484b-be5d-c074e1020907\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") "
Sep 11 12:06:55.045381 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.045242    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv7b5\" (UniqueName: \"kubernetes.io/projected/eb6304ea-12a8-484b-be5d-c074e1020907-kube-api-access-pv7b5\") pod \"eb6304ea-12a8-484b-be5d-c074e1020907\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") "
Sep 11 12:06:55.045381 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.045287    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-metrics-certs\") pod \"eb6304ea-12a8-484b-be5d-c074e1020907\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") "
Sep 11 12:06:55.045381 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.045366    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-maas-api-tls\") pod \"eb6304ea-12a8-484b-be5d-c074e1020907\" (UID: \"eb6304ea-12a8-484b-be5d-c074e1020907\") "
Sep 11 12:06:55.046142 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.046078    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb6304ea-12a8-484b-be5d-c074e1020907-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eb6304ea-12a8-484b-be5d-c074e1020907" (UID: "eb6304ea-12a8-484b-be5d-c074e1020907"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:06:55.048026 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.047997    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eb6304ea-12a8-484b-be5d-c074e1020907" (UID: "eb6304ea-12a8-484b-be5d-c074e1020907"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:06:55.048141 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.048117    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eb6304ea-12a8-484b-be5d-c074e1020907" (UID: "eb6304ea-12a8-484b-be5d-c074e1020907"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:06:55.048141 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.048131    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6304ea-12a8-484b-be5d-c074e1020907-kube-api-access-pv7b5" (OuterVolumeSpecName: "kube-api-access-pv7b5") pod "eb6304ea-12a8-484b-be5d-c074e1020907" (UID: "eb6304ea-12a8-484b-be5d-c074e1020907"). InnerVolumeSpecName "kube-api-access-pv7b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:55.056381 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.056353    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:06:55.146093 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.146052    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b752d36b-1b53-4f5c-9995-dbbc28e7c171-plugins-config-volume\") pod \"b752d36b-1b53-4f5c-9995-dbbc28e7c171\" (UID: \"b752d36b-1b53-4f5c-9995-dbbc28e7c171\") "
Sep 11 12:06:55.146288 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.146147    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frkwt\" (UniqueName: \"kubernetes.io/projected/b752d36b-1b53-4f5c-9995-dbbc28e7c171-kube-api-access-frkwt\") pod \"b752d36b-1b53-4f5c-9995-dbbc28e7c171\" (UID: \"b752d36b-1b53-4f5c-9995-dbbc28e7c171\") "
Sep 11 12:06:55.146498 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.146470    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b752d36b-1b53-4f5c-9995-dbbc28e7c171-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b752d36b-1b53-4f5c-9995-dbbc28e7c171" (UID: "b752d36b-1b53-4f5c-9995-dbbc28e7c171"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:06:55.146564 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.146484    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb6304ea-12a8-484b-be5d-c074e1020907-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:55.146564 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.146518    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv7b5\" (UniqueName: \"kubernetes.io/projected/eb6304ea-12a8-484b-be5d-c074e1020907-kube-api-access-pv7b5\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:55.146564 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.146532    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:55.146564 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.146545    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb6304ea-12a8-484b-be5d-c074e1020907-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:55.148115 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.148095    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b752d36b-1b53-4f5c-9995-dbbc28e7c171-kube-api-access-frkwt" (OuterVolumeSpecName: "kube-api-access-frkwt") pod "b752d36b-1b53-4f5c-9995-dbbc28e7c171" (UID: "b752d36b-1b53-4f5c-9995-dbbc28e7c171"). InnerVolumeSpecName "kube-api-access-frkwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:55.247620 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.247530    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b752d36b-1b53-4f5c-9995-dbbc28e7c171-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:55.247620 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.247559    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frkwt\" (UniqueName: \"kubernetes.io/projected/b752d36b-1b53-4f5c-9995-dbbc28e7c171-kube-api-access-frkwt\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:06:55.508745 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.508647    2562 generic.go:356] "Generic (PLEG): container finished" podID="b752d36b-1b53-4f5c-9995-dbbc28e7c171" containerID="f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3" exitCode=0
Sep 11 12:06:55.508745 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.508723    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"
Sep 11 12:06:55.509045 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.508740    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn" event={"ID":"b752d36b-1b53-4f5c-9995-dbbc28e7c171","Type":"ContainerDied","Data":"f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3"}
Sep 11 12:06:55.509045 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.508833    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn" event={"ID":"b752d36b-1b53-4f5c-9995-dbbc28e7c171","Type":"ContainerDied","Data":"5f3042d4daf38242563fdd4ac781d0e51f2de352806e809679f0372c48f624e4"}
Sep 11 12:06:55.509045 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.508866    2562 scope.go:117] "RemoveContainer" containerID="f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3"
Sep 11 12:06:55.510226 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.510196    2562 generic.go:356] "Generic (PLEG): container finished" podID="eb6304ea-12a8-484b-be5d-c074e1020907" containerID="90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82" exitCode=0
Sep 11 12:06:55.510360 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.510225    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb" event={"ID":"eb6304ea-12a8-484b-be5d-c074e1020907","Type":"ContainerDied","Data":"90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82"}
Sep 11 12:06:55.510360 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.510264    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"
Sep 11 12:06:55.510360 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.510282    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb" event={"ID":"eb6304ea-12a8-484b-be5d-c074e1020907","Type":"ContainerDied","Data":"3429cf4c31bcd74ff378c7db528b6c91e17133c898db8327ee008acb00d9cfd7"}
Sep 11 12:06:55.520140 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.520124    2562 scope.go:117] "RemoveContainer" containerID="f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3"
Sep 11 12:06:55.520504 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:06:55.520484    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3\": container with ID starting with f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3 not found: ID does not exist" containerID="f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3"
Sep 11 12:06:55.520564 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.520511    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3"} err="failed to get container status \"f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3\": rpc error: code = NotFound desc = could not find container \"f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3\": container with ID starting with f91b0691f7bcf26fbaeb04bf599153147b0ac8383a6968f41b36cd4417d190b3 not found: ID does not exist"
Sep 11 12:06:55.520564 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.520528    2562 scope.go:117] "RemoveContainer" containerID="90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82"
Sep 11 12:06:55.528868 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.528848    2562 scope.go:117] "RemoveContainer" containerID="90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82"
Sep 11 12:06:55.529154 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:06:55.529137    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82\": container with ID starting with 90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82 not found: ID does not exist" containerID="90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82"
Sep 11 12:06:55.529206 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.529159    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82"} err="failed to get container status \"90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82\": rpc error: code = NotFound desc = could not find container \"90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82\": container with ID starting with 90fa3b64e791c264e2d0199823c11479627297f1af66d908987a7983d16c0a82 not found: ID does not exist"
Sep 11 12:06:55.539328 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.539289    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"]
Sep 11 12:06:55.546309 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.546287    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2c2847-74b4b9677d-gzmbb"]
Sep 11 12:06:55.556139 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.556115    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"]
Sep 11 12:06:55.560542 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:55.560522    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c2847-745f55fdd6-fpcpn"]
Sep 11 12:06:56.027831 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:56.027787    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b752d36b-1b53-4f5c-9995-dbbc28e7c171" path="/var/lib/kubelet/pods/b752d36b-1b53-4f5c-9995-dbbc28e7c171/volumes"
Sep 11 12:06:56.028426 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:06:56.028405    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb6304ea-12a8-484b-be5d-c074e1020907" path="/var/lib/kubelet/pods/eb6304ea-12a8-484b-be5d-c074e1020907/volumes"
Sep 11 12:07:05.937563 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:05.937525    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"]
Sep 11 12:07:05.938028 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:05.937957    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b752d36b-1b53-4f5c-9995-dbbc28e7c171" containerName="payload-pre-processing"
Sep 11 12:07:05.938028 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:05.937968    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b752d36b-1b53-4f5c-9995-dbbc28e7c171" containerName="payload-pre-processing"
Sep 11 12:07:05.938028 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:05.937979    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb6304ea-12a8-484b-be5d-c074e1020907" containerName="maas-api"
Sep 11 12:07:05.938028 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:05.937985    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb6304ea-12a8-484b-be5d-c074e1020907" containerName="maas-api"
Sep 11 12:07:05.938161 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:05.938054    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb6304ea-12a8-484b-be5d-c074e1020907" containerName="maas-api"
Sep 11 12:07:05.938161 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:05.938063    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b752d36b-1b53-4f5c-9995-dbbc28e7c171" containerName="payload-pre-processing"
Sep 11 12:07:05.941285 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:05.941267    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:05.943418 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:05.943399    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-176ff0-serving-cert\""
Sep 11 12:07:05.949149 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:05.949125    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"]
Sep 11 12:07:06.051996 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.051962    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-metrics-certs\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.052174 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.052027    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqmb6\" (UniqueName: \"kubernetes.io/projected/4de37e83-ef51-48d3-b777-bb131b6275ba-kube-api-access-sqmb6\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.052174 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.052122    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de37e83-ef51-48d3-b777-bb131b6275ba-openshift-service-ca\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.052174 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.052162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.153146 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.153108    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqmb6\" (UniqueName: \"kubernetes.io/projected/4de37e83-ef51-48d3-b777-bb131b6275ba-kube-api-access-sqmb6\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.153355 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.153198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de37e83-ef51-48d3-b777-bb131b6275ba-openshift-service-ca\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.153355 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.153235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.153355 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.153263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-metrics-certs\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.153482 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:07:06.153388    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-serving-cert: secret "maas-api-e2e-explicit-176ff0-serving-cert" not found
Sep 11 12:07:06.153482 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:07:06.153454    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls podName:4de37e83-ef51-48d3-b777-bb131b6275ba nodeName:}" failed. No retries permitted until 2026-09-11 12:07:06.653438625 +0000 UTC m=+935.154158871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls") pod "maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m" (UID: "4de37e83-ef51-48d3-b777-bb131b6275ba") : secret "maas-api-e2e-explicit-176ff0-serving-cert" not found
Sep 11 12:07:06.153869 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.153851    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de37e83-ef51-48d3-b777-bb131b6275ba-openshift-service-ca\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.155724 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.155706    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-metrics-certs\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.161377 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.161356    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqmb6\" (UniqueName: \"kubernetes.io/projected/4de37e83-ef51-48d3-b777-bb131b6275ba-kube-api-access-sqmb6\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.657398 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:06.657357    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:06.657572 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:07:06.657515    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-serving-cert: secret "maas-api-e2e-explicit-176ff0-serving-cert" not found
Sep 11 12:07:06.657615 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:07:06.657582    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls podName:4de37e83-ef51-48d3-b777-bb131b6275ba nodeName:}" failed. No retries permitted until 2026-09-11 12:07:07.65756623 +0000 UTC m=+936.158286481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls") pod "maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m" (UID: "4de37e83-ef51-48d3-b777-bb131b6275ba") : secret "maas-api-e2e-explicit-176ff0-serving-cert" not found
Sep 11 12:07:07.666818 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:07.666771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:07.669227 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:07.669193    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls\") pod \"maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:07.753352 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:07.753288    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:08.031888 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:08.031859    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"]
Sep 11 12:07:08.034608 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:07:08.034580    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de37e83_ef51_48d3_b777_bb131b6275ba.slice/crio-5396a35c972e645ede8d1b88402f2b70f51dfd9126e23f26c81486eb4d8a40e9 WatchSource:0}: Error finding container 5396a35c972e645ede8d1b88402f2b70f51dfd9126e23f26c81486eb4d8a40e9: Status 404 returned error can't find the container with id 5396a35c972e645ede8d1b88402f2b70f51dfd9126e23f26c81486eb4d8a40e9
Sep 11 12:07:08.581920 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:08.581879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m" event={"ID":"4de37e83-ef51-48d3-b777-bb131b6275ba","Type":"ContainerStarted","Data":"a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021"}
Sep 11 12:07:08.582127 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:08.581937    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m" event={"ID":"4de37e83-ef51-48d3-b777-bb131b6275ba","Type":"ContainerStarted","Data":"5396a35c972e645ede8d1b88402f2b70f51dfd9126e23f26c81486eb4d8a40e9"}
Sep 11 12:07:08.582127 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:08.582022    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:07:08.607971 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:08.607852    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m" podStartSLOduration=3.291099327 podStartE2EDuration="3.607835993s" totalImagesPullingTime="316.736666ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:07:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:07:08.604682747 +0000 UTC m=+937.105403016" watchObservedRunningTime="2026-09-11 12:07:08.607835993 +0000 UTC m=+937.108556261"
Sep 11 12:07:14.593145 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:07:14.593114    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:08:07.330607 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.330519    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"]
Sep 11 12:08:07.335083 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.335055    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.337742 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.337717    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5e9d4e44-serving-cert\""
Sep 11 12:08:07.343625 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.343593    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"]
Sep 11 12:08:07.347106 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.347074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htqwc\" (UniqueName: \"kubernetes.io/projected/f46e2e54-9a98-4137-a590-93135f7f7dd8-kube-api-access-htqwc\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.347242 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.347156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f46e2e54-9a98-4137-a590-93135f7f7dd8-openshift-service-ca\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.347242 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.347199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-maas-api-tls\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.347378 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.347243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-metrics-certs\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.448555 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.448515    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f46e2e54-9a98-4137-a590-93135f7f7dd8-openshift-service-ca\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.448780 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.448574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-maas-api-tls\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.448780 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.448624    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-metrics-certs\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.448780 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:08:07.448729    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-serving-cert: secret "maas-api-e2e-mt-5e9d4e44-serving-cert" not found
Sep 11 12:08:07.448977 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.448782    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htqwc\" (UniqueName: \"kubernetes.io/projected/f46e2e54-9a98-4137-a590-93135f7f7dd8-kube-api-access-htqwc\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.448977 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:08:07.448850    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-maas-api-tls podName:f46e2e54-9a98-4137-a590-93135f7f7dd8 nodeName:}" failed. No retries permitted until 2026-09-11 12:08:07.948826652 +0000 UTC m=+996.449546910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-maas-api-tls") pod "maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22" (UID: "f46e2e54-9a98-4137-a590-93135f7f7dd8") : secret "maas-api-e2e-mt-5e9d4e44-serving-cert" not found
Sep 11 12:08:07.449350 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.449325    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f46e2e54-9a98-4137-a590-93135f7f7dd8-openshift-service-ca\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.451450 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.451429    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-metrics-certs\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.460028 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.460008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htqwc\" (UniqueName: \"kubernetes.io/projected/f46e2e54-9a98-4137-a590-93135f7f7dd8-kube-api-access-htqwc\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.954717 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.954673    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-maas-api-tls\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.957230 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.957204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-maas-api-tls\") pod \"maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:07.983924 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.983884    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"]
Sep 11 12:08:07.987656 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.987638    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:07.989889 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.989867    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-5e9d4e44-dockercfg-w56pq\""
Sep 11 12:08:07.990390 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.990366    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-5e9d4e44\""
Sep 11 12:08:07.997870 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:07.997845    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"]
Sep 11 12:08:08.011613 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.011579    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"]
Sep 11 12:08:08.015003 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.014985    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:08.027928 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.027896    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"]
Sep 11 12:08:08.055947 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.055903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/600181af-041a-421b-9a90-63b012780dac-plugins-config-volume\") pod \"payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns\" (UID: \"600181af-041a-421b-9a90-63b012780dac\") " pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:08.056128 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.055954    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csn54\" (UniqueName: \"kubernetes.io/projected/4ccdcdd2-4b16-454c-b479-bc772c24e046-kube-api-access-csn54\") pod \"payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr\" (UID: \"4ccdcdd2-4b16-454c-b479-bc772c24e046\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:08.056128 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.056062    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qltg5\" (UniqueName: \"kubernetes.io/projected/600181af-041a-421b-9a90-63b012780dac-kube-api-access-qltg5\") pod \"payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns\" (UID: \"600181af-041a-421b-9a90-63b012780dac\") " pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:08.056212 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.056152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ccdcdd2-4b16-454c-b479-bc772c24e046-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr\" (UID: \"4ccdcdd2-4b16-454c-b479-bc772c24e046\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:08.157165 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.157128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ccdcdd2-4b16-454c-b479-bc772c24e046-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr\" (UID: \"4ccdcdd2-4b16-454c-b479-bc772c24e046\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:08.157381 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.157176    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/600181af-041a-421b-9a90-63b012780dac-plugins-config-volume\") pod \"payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns\" (UID: \"600181af-041a-421b-9a90-63b012780dac\") " pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:08.157381 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.157213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csn54\" (UniqueName: \"kubernetes.io/projected/4ccdcdd2-4b16-454c-b479-bc772c24e046-kube-api-access-csn54\") pod \"payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr\" (UID: \"4ccdcdd2-4b16-454c-b479-bc772c24e046\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:08.157381 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.157265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qltg5\" (UniqueName: \"kubernetes.io/projected/600181af-041a-421b-9a90-63b012780dac-kube-api-access-qltg5\") pod \"payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns\" (UID: \"600181af-041a-421b-9a90-63b012780dac\") " pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:08.157963 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.157936    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/600181af-041a-421b-9a90-63b012780dac-plugins-config-volume\") pod \"payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns\" (UID: \"600181af-041a-421b-9a90-63b012780dac\") " pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:08.158059 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.157946    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ccdcdd2-4b16-454c-b479-bc772c24e046-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr\" (UID: \"4ccdcdd2-4b16-454c-b479-bc772c24e046\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:08.164673 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.164640    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csn54\" (UniqueName: \"kubernetes.io/projected/4ccdcdd2-4b16-454c-b479-bc772c24e046-kube-api-access-csn54\") pod \"payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr\" (UID: \"4ccdcdd2-4b16-454c-b479-bc772c24e046\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:08.164786 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.164740    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qltg5\" (UniqueName: \"kubernetes.io/projected/600181af-041a-421b-9a90-63b012780dac-kube-api-access-qltg5\") pod \"payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns\" (UID: \"600181af-041a-421b-9a90-63b012780dac\") " pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:08.249946 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.249836    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:08.300175 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.300002    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:08.328126 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.328086    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:08.527275 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.527248    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"]
Sep 11 12:08:08.530095 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:08:08.530067    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf46e2e54_9a98_4137_a590_93135f7f7dd8.slice/crio-c9286994d05082273392b16d5d32845042db8a9f5a11386238bac123416f6023 WatchSource:0}: Error finding container c9286994d05082273392b16d5d32845042db8a9f5a11386238bac123416f6023: Status 404 returned error can't find the container with id c9286994d05082273392b16d5d32845042db8a9f5a11386238bac123416f6023
Sep 11 12:08:08.593954 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.593919    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"]
Sep 11 12:08:08.597563 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:08:08.597524    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ccdcdd2_4b16_454c_b479_bc772c24e046.slice/crio-d886a6760f3aafeb84c3e03877df6abb94671b60eacc9131665d553093588136 WatchSource:0}: Error finding container d886a6760f3aafeb84c3e03877df6abb94671b60eacc9131665d553093588136: Status 404 returned error can't find the container with id d886a6760f3aafeb84c3e03877df6abb94671b60eacc9131665d553093588136
Sep 11 12:08:08.625461 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.624598    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"]
Sep 11 12:08:08.627563 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:08:08.627529    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600181af_041a_421b_9a90_63b012780dac.slice/crio-5345d65388d3e1243cf02cb314cb21db3693a6e1cb15684e0e9369a33bf61a63 WatchSource:0}: Error finding container 5345d65388d3e1243cf02cb314cb21db3693a6e1cb15684e0e9369a33bf61a63: Status 404 returned error can't find the container with id 5345d65388d3e1243cf02cb314cb21db3693a6e1cb15684e0e9369a33bf61a63
Sep 11 12:08:08.894859 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.894757    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22" event={"ID":"f46e2e54-9a98-4137-a590-93135f7f7dd8","Type":"ContainerStarted","Data":"c9286994d05082273392b16d5d32845042db8a9f5a11386238bac123416f6023"}
Sep 11 12:08:08.896507 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.896478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr" event={"ID":"4ccdcdd2-4b16-454c-b479-bc772c24e046","Type":"ContainerStarted","Data":"55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867"}
Sep 11 12:08:08.896665 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.896525    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr" event={"ID":"4ccdcdd2-4b16-454c-b479-bc772c24e046","Type":"ContainerStarted","Data":"d886a6760f3aafeb84c3e03877df6abb94671b60eacc9131665d553093588136"}
Sep 11 12:08:08.896665 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.896569    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:08.898175 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.898153    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns" event={"ID":"600181af-041a-421b-9a90-63b012780dac","Type":"ContainerStarted","Data":"276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60"}
Sep 11 12:08:08.898298 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.898195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns" event={"ID":"600181af-041a-421b-9a90-63b012780dac","Type":"ContainerStarted","Data":"5345d65388d3e1243cf02cb314cb21db3693a6e1cb15684e0e9369a33bf61a63"}
Sep 11 12:08:08.898298 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.898234    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:08.914710 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.914657    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr" podStartSLOduration=1.9146371420000001 podStartE2EDuration="1.914637142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:08.911345474 +0000 UTC m=+997.412065743" watchObservedRunningTime="2026-09-11 12:08:08.914637142 +0000 UTC m=+997.415357413"
Sep 11 12:08:08.927681 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:08.927630    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns" podStartSLOduration=1.92761562 podStartE2EDuration="1.92761562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:08.926210953 +0000 UTC m=+997.426931222" watchObservedRunningTime="2026-09-11 12:08:08.92761562 +0000 UTC m=+997.428335888"
Sep 11 12:08:09.906182 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:09.906141    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22" event={"ID":"f46e2e54-9a98-4137-a590-93135f7f7dd8","Type":"ContainerStarted","Data":"52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d"}
Sep 11 12:08:09.906693 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:09.906286    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:09.923744 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:09.923696    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22" podStartSLOduration=2.457372708 podStartE2EDuration="2.923680166s" totalImagesPullingTime="466.307458ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:09.922918463 +0000 UTC m=+998.423638757" watchObservedRunningTime="2026-09-11 12:08:09.923680166 +0000 UTC m=+998.424400769"
Sep 11 12:08:15.917731 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:15.917692    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:19.908976 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:19.908944    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:19.909434 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:19.909173    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:32.509777 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.509741    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"]
Sep 11 12:08:32.510190 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.509964    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m" podUID="4de37e83-ef51-48d3-b777-bb131b6275ba" containerName="maas-api" containerID="cri-o://a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021" gracePeriod=30
Sep 11 12:08:32.772944 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.772879    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:08:32.905558 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.905519    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls\") pod \"4de37e83-ef51-48d3-b777-bb131b6275ba\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") "
Sep 11 12:08:32.905724 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.905610    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqmb6\" (UniqueName: \"kubernetes.io/projected/4de37e83-ef51-48d3-b777-bb131b6275ba-kube-api-access-sqmb6\") pod \"4de37e83-ef51-48d3-b777-bb131b6275ba\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") "
Sep 11 12:08:32.905724 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.905672    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-metrics-certs\") pod \"4de37e83-ef51-48d3-b777-bb131b6275ba\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") "
Sep 11 12:08:32.905724 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.905704    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de37e83-ef51-48d3-b777-bb131b6275ba-openshift-service-ca\") pod \"4de37e83-ef51-48d3-b777-bb131b6275ba\" (UID: \"4de37e83-ef51-48d3-b777-bb131b6275ba\") "
Sep 11 12:08:32.906119 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.906093    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de37e83-ef51-48d3-b777-bb131b6275ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4de37e83-ef51-48d3-b777-bb131b6275ba" (UID: "4de37e83-ef51-48d3-b777-bb131b6275ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:08:32.907754 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.907724    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4de37e83-ef51-48d3-b777-bb131b6275ba" (UID: "4de37e83-ef51-48d3-b777-bb131b6275ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:08:32.907754 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.907743    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4de37e83-ef51-48d3-b777-bb131b6275ba" (UID: "4de37e83-ef51-48d3-b777-bb131b6275ba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:08:32.907885 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:32.907750    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de37e83-ef51-48d3-b777-bb131b6275ba-kube-api-access-sqmb6" (OuterVolumeSpecName: "kube-api-access-sqmb6") pod "4de37e83-ef51-48d3-b777-bb131b6275ba" (UID: "4de37e83-ef51-48d3-b777-bb131b6275ba"). InnerVolumeSpecName "kube-api-access-sqmb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:33.007288 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.007245    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqmb6\" (UniqueName: \"kubernetes.io/projected/4de37e83-ef51-48d3-b777-bb131b6275ba-kube-api-access-sqmb6\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:33.007480 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.007336    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:33.007480 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.007354    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de37e83-ef51-48d3-b777-bb131b6275ba-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:33.007480 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.007363    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4de37e83-ef51-48d3-b777-bb131b6275ba-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:33.033247 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.033152    2562 generic.go:356] "Generic (PLEG): container finished" podID="4de37e83-ef51-48d3-b777-bb131b6275ba" containerID="a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021" exitCode=0
Sep 11 12:08:33.033247 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.033176    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m" event={"ID":"4de37e83-ef51-48d3-b777-bb131b6275ba","Type":"ContainerDied","Data":"a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021"}
Sep 11 12:08:33.033247 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.033206    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"
Sep 11 12:08:33.033247 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.033230    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m" event={"ID":"4de37e83-ef51-48d3-b777-bb131b6275ba","Type":"ContainerDied","Data":"5396a35c972e645ede8d1b88402f2b70f51dfd9126e23f26c81486eb4d8a40e9"}
Sep 11 12:08:33.033558 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.033263    2562 scope.go:117] "RemoveContainer" containerID="a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021"
Sep 11 12:08:33.043455 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.043435    2562 scope.go:117] "RemoveContainer" containerID="a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021"
Sep 11 12:08:33.043737 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:08:33.043719    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021\": container with ID starting with a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021 not found: ID does not exist" containerID="a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021"
Sep 11 12:08:33.043805 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.043744    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021"} err="failed to get container status \"a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021\": rpc error: code = NotFound desc = could not find container \"a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021\": container with ID starting with a693169e284e87acbb660a73cf94b30ecfe3d6036f84123cda781db5949af021 not found: ID does not exist"
Sep 11 12:08:33.058474 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.058451    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"]
Sep 11 12:08:33.062467 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:33.062444    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-176ff0-99c87c4f5-r8b5m"]
Sep 11 12:08:34.025153 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:34.025123    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4de37e83-ef51-48d3-b777-bb131b6275ba" path="/var/lib/kubelet/pods/4de37e83-ef51-48d3-b777-bb131b6275ba/volumes"
Sep 11 12:08:38.520357 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:38.520287    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"]
Sep 11 12:08:49.313507 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.313471    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"]
Sep 11 12:08:49.314083 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.313700    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22" podUID="f46e2e54-9a98-4137-a590-93135f7f7dd8" containerName="maas-api" containerID="cri-o://52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d" gracePeriod=30
Sep 11 12:08:49.396326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.396273    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"]
Sep 11 12:08:49.396631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.396571    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns" podUID="600181af-041a-421b-9a90-63b012780dac" containerName="payload-processing" containerID="cri-o://276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60" gracePeriod=30
Sep 11 12:08:49.401757 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.401728    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"]
Sep 11 12:08:49.402014 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.401989    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr" podUID="4ccdcdd2-4b16-454c-b479-bc772c24e046" containerName="payload-pre-processing" containerID="cri-o://55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867" gracePeriod=30
Sep 11 12:08:49.659781 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.659751    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:49.687386 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.687359    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:49.723782 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.723751    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:49.770114 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770073    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f46e2e54-9a98-4137-a590-93135f7f7dd8-openshift-service-ca\") pod \"f46e2e54-9a98-4137-a590-93135f7f7dd8\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") "
Sep 11 12:08:49.770318 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770136    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htqwc\" (UniqueName: \"kubernetes.io/projected/f46e2e54-9a98-4137-a590-93135f7f7dd8-kube-api-access-htqwc\") pod \"f46e2e54-9a98-4137-a590-93135f7f7dd8\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") "
Sep 11 12:08:49.770318 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770165    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-metrics-certs\") pod \"f46e2e54-9a98-4137-a590-93135f7f7dd8\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") "
Sep 11 12:08:49.770318 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770183    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ccdcdd2-4b16-454c-b479-bc772c24e046-plugins-config-volume\") pod \"4ccdcdd2-4b16-454c-b479-bc772c24e046\" (UID: \"4ccdcdd2-4b16-454c-b479-bc772c24e046\") "
Sep 11 12:08:49.770318 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770212    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/600181af-041a-421b-9a90-63b012780dac-plugins-config-volume\") pod \"600181af-041a-421b-9a90-63b012780dac\" (UID: \"600181af-041a-421b-9a90-63b012780dac\") "
Sep 11 12:08:49.770318 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770227    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qltg5\" (UniqueName: \"kubernetes.io/projected/600181af-041a-421b-9a90-63b012780dac-kube-api-access-qltg5\") pod \"600181af-041a-421b-9a90-63b012780dac\" (UID: \"600181af-041a-421b-9a90-63b012780dac\") "
Sep 11 12:08:49.770318 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770269    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csn54\" (UniqueName: \"kubernetes.io/projected/4ccdcdd2-4b16-454c-b479-bc772c24e046-kube-api-access-csn54\") pod \"4ccdcdd2-4b16-454c-b479-bc772c24e046\" (UID: \"4ccdcdd2-4b16-454c-b479-bc772c24e046\") "
Sep 11 12:08:49.770662 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770362    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-maas-api-tls\") pod \"f46e2e54-9a98-4137-a590-93135f7f7dd8\" (UID: \"f46e2e54-9a98-4137-a590-93135f7f7dd8\") "
Sep 11 12:08:49.770662 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770543    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f46e2e54-9a98-4137-a590-93135f7f7dd8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f46e2e54-9a98-4137-a590-93135f7f7dd8" (UID: "f46e2e54-9a98-4137-a590-93135f7f7dd8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:08:49.770662 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770597    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/600181af-041a-421b-9a90-63b012780dac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "600181af-041a-421b-9a90-63b012780dac" (UID: "600181af-041a-421b-9a90-63b012780dac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:08:49.770805 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770763    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/600181af-041a-421b-9a90-63b012780dac-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:49.770805 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770785    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f46e2e54-9a98-4137-a590-93135f7f7dd8-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:49.770931 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.770866    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ccdcdd2-4b16-454c-b479-bc772c24e046-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ccdcdd2-4b16-454c-b479-bc772c24e046" (UID: "4ccdcdd2-4b16-454c-b479-bc772c24e046"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:08:49.772481 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.772456    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f46e2e54-9a98-4137-a590-93135f7f7dd8-kube-api-access-htqwc" (OuterVolumeSpecName: "kube-api-access-htqwc") pod "f46e2e54-9a98-4137-a590-93135f7f7dd8" (UID: "f46e2e54-9a98-4137-a590-93135f7f7dd8"). InnerVolumeSpecName "kube-api-access-htqwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:49.772576 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.772511    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600181af-041a-421b-9a90-63b012780dac-kube-api-access-qltg5" (OuterVolumeSpecName: "kube-api-access-qltg5") pod "600181af-041a-421b-9a90-63b012780dac" (UID: "600181af-041a-421b-9a90-63b012780dac"). InnerVolumeSpecName "kube-api-access-qltg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:49.772759 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.772744    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f46e2e54-9a98-4137-a590-93135f7f7dd8" (UID: "f46e2e54-9a98-4137-a590-93135f7f7dd8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:08:49.772967 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.772944    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ccdcdd2-4b16-454c-b479-bc772c24e046-kube-api-access-csn54" (OuterVolumeSpecName: "kube-api-access-csn54") pod "4ccdcdd2-4b16-454c-b479-bc772c24e046" (UID: "4ccdcdd2-4b16-454c-b479-bc772c24e046"). InnerVolumeSpecName "kube-api-access-csn54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:49.772967 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.772959    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f46e2e54-9a98-4137-a590-93135f7f7dd8" (UID: "f46e2e54-9a98-4137-a590-93135f7f7dd8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:08:49.871474 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.871391    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:49.871474 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.871420    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htqwc\" (UniqueName: \"kubernetes.io/projected/f46e2e54-9a98-4137-a590-93135f7f7dd8-kube-api-access-htqwc\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:49.871474 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.871429    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f46e2e54-9a98-4137-a590-93135f7f7dd8-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:49.871474 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.871439    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ccdcdd2-4b16-454c-b479-bc772c24e046-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:49.871474 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.871446    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qltg5\" (UniqueName: \"kubernetes.io/projected/600181af-041a-421b-9a90-63b012780dac-kube-api-access-qltg5\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:49.871474 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:49.871455    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csn54\" (UniqueName: \"kubernetes.io/projected/4ccdcdd2-4b16-454c-b479-bc772c24e046-kube-api-access-csn54\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:50.120778 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.120739    2562 generic.go:356] "Generic (PLEG): container finished" podID="600181af-041a-421b-9a90-63b012780dac" containerID="276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60" exitCode=0
Sep 11 12:08:50.120978 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.120786    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"
Sep 11 12:08:50.120978 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.120818    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns" event={"ID":"600181af-041a-421b-9a90-63b012780dac","Type":"ContainerDied","Data":"276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60"}
Sep 11 12:08:50.120978 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.120876    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns" event={"ID":"600181af-041a-421b-9a90-63b012780dac","Type":"ContainerDied","Data":"5345d65388d3e1243cf02cb314cb21db3693a6e1cb15684e0e9369a33bf61a63"}
Sep 11 12:08:50.120978 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.120905    2562 scope.go:117] "RemoveContainer" containerID="276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60"
Sep 11 12:08:50.122349 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.122321    2562 generic.go:356] "Generic (PLEG): container finished" podID="f46e2e54-9a98-4137-a590-93135f7f7dd8" containerID="52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d" exitCode=0
Sep 11 12:08:50.122472 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.122359    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22" event={"ID":"f46e2e54-9a98-4137-a590-93135f7f7dd8","Type":"ContainerDied","Data":"52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d"}
Sep 11 12:08:50.122472 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.122384    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"
Sep 11 12:08:50.122472 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.122416    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22" event={"ID":"f46e2e54-9a98-4137-a590-93135f7f7dd8","Type":"ContainerDied","Data":"c9286994d05082273392b16d5d32845042db8a9f5a11386238bac123416f6023"}
Sep 11 12:08:50.123974 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.123953    2562 generic.go:356] "Generic (PLEG): container finished" podID="4ccdcdd2-4b16-454c-b479-bc772c24e046" containerID="55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867" exitCode=0
Sep 11 12:08:50.124079 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.123988    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"
Sep 11 12:08:50.124079 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.124006    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr" event={"ID":"4ccdcdd2-4b16-454c-b479-bc772c24e046","Type":"ContainerDied","Data":"55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867"}
Sep 11 12:08:50.124079 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.124047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr" event={"ID":"4ccdcdd2-4b16-454c-b479-bc772c24e046","Type":"ContainerDied","Data":"d886a6760f3aafeb84c3e03877df6abb94671b60eacc9131665d553093588136"}
Sep 11 12:08:50.130822 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.130802    2562 scope.go:117] "RemoveContainer" containerID="276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60"
Sep 11 12:08:50.131118 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:08:50.131098    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60\": container with ID starting with 276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60 not found: ID does not exist" containerID="276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60"
Sep 11 12:08:50.131183 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.131126    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60"} err="failed to get container status \"276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60\": rpc error: code = NotFound desc = could not find container \"276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60\": container with ID starting with 276af5d187db675ae39ceffd1d4822f18d2e961a34fe03b6e2d4c851063ebd60 not found: ID does not exist"
Sep 11 12:08:50.131183 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.131142    2562 scope.go:117] "RemoveContainer" containerID="52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d"
Sep 11 12:08:50.139101 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.139079    2562 scope.go:117] "RemoveContainer" containerID="52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d"
Sep 11 12:08:50.139381 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:08:50.139364    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d\": container with ID starting with 52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d not found: ID does not exist" containerID="52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d"
Sep 11 12:08:50.139445 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.139388    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d"} err="failed to get container status \"52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d\": rpc error: code = NotFound desc = could not find container \"52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d\": container with ID starting with 52bb80d4ad6abc83f6c144ed42e985c12c59076f0cfcc9203491545d57bfe33d not found: ID does not exist"
Sep 11 12:08:50.139445 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.139413    2562 scope.go:117] "RemoveContainer" containerID="55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867"
Sep 11 12:08:50.144388 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.144362    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"]
Sep 11 12:08:50.147281 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.147261    2562 scope.go:117] "RemoveContainer" containerID="55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867"
Sep 11 12:08:50.147629 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:08:50.147605    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867\": container with ID starting with 55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867 not found: ID does not exist" containerID="55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867"
Sep 11 12:08:50.147689 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.147637    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867"} err="failed to get container status \"55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867\": rpc error: code = NotFound desc = could not find container \"55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867\": container with ID starting with 55d6325fe87e474d7bb600a837d749d14f51321356e5aa75e048e9c969bfa867 not found: ID does not exist"
Sep 11 12:08:50.150113 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.150090    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e9d4e44-8b58db99b-wph22"]
Sep 11 12:08:50.160071 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.160050    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"]
Sep 11 12:08:50.161921 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.161901    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e9d4e44-867f69667c-ggbns"]
Sep 11 12:08:50.171659 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.171633    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"]
Sep 11 12:08:50.178250 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:50.178225    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e9d4e44-5ccd998c4b-2nqsr"]
Sep 11 12:08:51.069883 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.069850    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"]
Sep 11 12:08:51.070382 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.070105    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm" podUID="48661d42-238f-47ec-8c27-3d8b8b0ff8db" containerName="maas-api" containerID="cri-o://115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8" gracePeriod=30
Sep 11 12:08:51.145217 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.145184    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"]
Sep 11 12:08:51.145496 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.145462    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k" podUID="fa5c5396-81ea-4410-b5da-102c1281bc43" containerName="payload-pre-processing" containerID="cri-o://2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7" gracePeriod=30
Sep 11 12:08:51.413658 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.413609    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:08:51.488345 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.488281    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5c5396-81ea-4410-b5da-102c1281bc43-plugins-config-volume\") pod \"fa5c5396-81ea-4410-b5da-102c1281bc43\" (UID: \"fa5c5396-81ea-4410-b5da-102c1281bc43\") "
Sep 11 12:08:51.488527 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.488380    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87w58\" (UniqueName: \"kubernetes.io/projected/fa5c5396-81ea-4410-b5da-102c1281bc43-kube-api-access-87w58\") pod \"fa5c5396-81ea-4410-b5da-102c1281bc43\" (UID: \"fa5c5396-81ea-4410-b5da-102c1281bc43\") "
Sep 11 12:08:51.488827 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.488799    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa5c5396-81ea-4410-b5da-102c1281bc43-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa5c5396-81ea-4410-b5da-102c1281bc43" (UID: "fa5c5396-81ea-4410-b5da-102c1281bc43"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:08:51.491006 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.490981    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa5c5396-81ea-4410-b5da-102c1281bc43-kube-api-access-87w58" (OuterVolumeSpecName: "kube-api-access-87w58") pod "fa5c5396-81ea-4410-b5da-102c1281bc43" (UID: "fa5c5396-81ea-4410-b5da-102c1281bc43"). InnerVolumeSpecName "kube-api-access-87w58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:51.541920 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.541894    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:08:51.589102 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.589065    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-maas-api-tls\") pod \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") "
Sep 11 12:08:51.589268 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.589144    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-metrics-certs\") pod \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") "
Sep 11 12:08:51.589268 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.589205    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48661d42-238f-47ec-8c27-3d8b8b0ff8db-openshift-service-ca\") pod \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") "
Sep 11 12:08:51.589386 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.589275    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz9j8\" (UniqueName: \"kubernetes.io/projected/48661d42-238f-47ec-8c27-3d8b8b0ff8db-kube-api-access-kz9j8\") pod \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\" (UID: \"48661d42-238f-47ec-8c27-3d8b8b0ff8db\") "
Sep 11 12:08:51.589586 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.589566    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5c5396-81ea-4410-b5da-102c1281bc43-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:51.589586 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.589587    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87w58\" (UniqueName: \"kubernetes.io/projected/fa5c5396-81ea-4410-b5da-102c1281bc43-kube-api-access-87w58\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:51.589761 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.589603    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48661d42-238f-47ec-8c27-3d8b8b0ff8db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "48661d42-238f-47ec-8c27-3d8b8b0ff8db" (UID: "48661d42-238f-47ec-8c27-3d8b8b0ff8db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:08:51.591356 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.591332    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "48661d42-238f-47ec-8c27-3d8b8b0ff8db" (UID: "48661d42-238f-47ec-8c27-3d8b8b0ff8db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:08:51.591616 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.591589    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "48661d42-238f-47ec-8c27-3d8b8b0ff8db" (UID: "48661d42-238f-47ec-8c27-3d8b8b0ff8db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:08:51.591616 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.591606    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48661d42-238f-47ec-8c27-3d8b8b0ff8db-kube-api-access-kz9j8" (OuterVolumeSpecName: "kube-api-access-kz9j8") pod "48661d42-238f-47ec-8c27-3d8b8b0ff8db" (UID: "48661d42-238f-47ec-8c27-3d8b8b0ff8db"). InnerVolumeSpecName "kube-api-access-kz9j8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:51.690159 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.690122    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz9j8\" (UniqueName: \"kubernetes.io/projected/48661d42-238f-47ec-8c27-3d8b8b0ff8db-kube-api-access-kz9j8\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:51.690159 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.690149    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:51.690159 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.690160    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48661d42-238f-47ec-8c27-3d8b8b0ff8db-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:51.690460 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:51.690174    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48661d42-238f-47ec-8c27-3d8b8b0ff8db-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:52.025176 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.025087    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ccdcdd2-4b16-454c-b479-bc772c24e046" path="/var/lib/kubelet/pods/4ccdcdd2-4b16-454c-b479-bc772c24e046/volumes"
Sep 11 12:08:52.025487 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.025472    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="600181af-041a-421b-9a90-63b012780dac" path="/var/lib/kubelet/pods/600181af-041a-421b-9a90-63b012780dac/volumes"
Sep 11 12:08:52.025819 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.025807    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f46e2e54-9a98-4137-a590-93135f7f7dd8" path="/var/lib/kubelet/pods/f46e2e54-9a98-4137-a590-93135f7f7dd8/volumes"
Sep 11 12:08:52.139775 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.139732    2562 generic.go:356] "Generic (PLEG): container finished" podID="48661d42-238f-47ec-8c27-3d8b8b0ff8db" containerID="115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8" exitCode=0
Sep 11 12:08:52.140251 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.139791    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"
Sep 11 12:08:52.140251 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.139817    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm" event={"ID":"48661d42-238f-47ec-8c27-3d8b8b0ff8db","Type":"ContainerDied","Data":"115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8"}
Sep 11 12:08:52.140251 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.139868    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm" event={"ID":"48661d42-238f-47ec-8c27-3d8b8b0ff8db","Type":"ContainerDied","Data":"a49ad0153f62ea9381647fe215bbfe1c58761e0cbd498b0127fca41b8584d965"}
Sep 11 12:08:52.140251 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.139896    2562 scope.go:117] "RemoveContainer" containerID="115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8"
Sep 11 12:08:52.141233 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.141211    2562 generic.go:356] "Generic (PLEG): container finished" podID="fa5c5396-81ea-4410-b5da-102c1281bc43" containerID="2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7" exitCode=0
Sep 11 12:08:52.141373 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.141269    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"
Sep 11 12:08:52.141373 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.141282    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k" event={"ID":"fa5c5396-81ea-4410-b5da-102c1281bc43","Type":"ContainerDied","Data":"2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7"}
Sep 11 12:08:52.141373 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.141345    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k" event={"ID":"fa5c5396-81ea-4410-b5da-102c1281bc43","Type":"ContainerDied","Data":"6c98f92046451c73199e74738fa2d5e5438451ff7822a419da54723a2af1c44b"}
Sep 11 12:08:52.148855 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.148833    2562 scope.go:117] "RemoveContainer" containerID="115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8"
Sep 11 12:08:52.149199 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:08:52.149171    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8\": container with ID starting with 115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8 not found: ID does not exist" containerID="115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8"
Sep 11 12:08:52.149324 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.149208    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8"} err="failed to get container status \"115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8\": rpc error: code = NotFound desc = could not find container \"115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8\": container with ID starting with 115ca05c4ae092016cdd7c622c460916bdbd7bbec7c9ab4e99b1cbdfd9b80aa8 not found: ID does not exist"
Sep 11 12:08:52.149324 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.149228    2562 scope.go:117] "RemoveContainer" containerID="2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7"
Sep 11 12:08:52.159838 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.158965    2562 scope.go:117] "RemoveContainer" containerID="2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7"
Sep 11 12:08:52.159838 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:08:52.159521    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7\": container with ID starting with 2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7 not found: ID does not exist" containerID="2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7"
Sep 11 12:08:52.159838 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.159560    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7"} err="failed to get container status \"2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7\": rpc error: code = NotFound desc = could not find container \"2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7\": container with ID starting with 2e4660c2e1894ebc6ceb3b65a7bb03c01c4e0d180efdbc133a5c5aae6c3fd5c7 not found: ID does not exist"
Sep 11 12:08:52.161981 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.161915    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"]
Sep 11 12:08:52.164996 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.164975    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34511f-54d5bb7974-hwj7k"]
Sep 11 12:08:52.178221 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.178195    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"]
Sep 11 12:08:52.184107 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:52.184081    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34511f-79d989dfd6-vwtfm"]
Sep 11 12:08:53.539508 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.539465    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp" podUID="9dbca512-ea16-4154-8a79-922a4780be51" containerName="main" containerID="cri-o://c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66" gracePeriod=45
Sep 11 12:08:53.792499 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.792441    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:08:53.910815 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.910779    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-dshm\") pod \"9dbca512-ea16-4154-8a79-922a4780be51\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") "
Sep 11 12:08:53.910815 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.910815    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-home\") pod \"9dbca512-ea16-4154-8a79-922a4780be51\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") "
Sep 11 12:08:53.911076 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.910839    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-tmp-dir\") pod \"9dbca512-ea16-4154-8a79-922a4780be51\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") "
Sep 11 12:08:53.911076 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.910882    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzq6t\" (UniqueName: \"kubernetes.io/projected/9dbca512-ea16-4154-8a79-922a4780be51-kube-api-access-jzq6t\") pod \"9dbca512-ea16-4154-8a79-922a4780be51\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") "
Sep 11 12:08:53.911076 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.910906    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-model-cache\") pod \"9dbca512-ea16-4154-8a79-922a4780be51\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") "
Sep 11 12:08:53.911076 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.910983    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9dbca512-ea16-4154-8a79-922a4780be51-tls-certs\") pod \"9dbca512-ea16-4154-8a79-922a4780be51\" (UID: \"9dbca512-ea16-4154-8a79-922a4780be51\") "
Sep 11 12:08:53.911299 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.911145    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-home" (OuterVolumeSpecName: "home") pod "9dbca512-ea16-4154-8a79-922a4780be51" (UID: "9dbca512-ea16-4154-8a79-922a4780be51"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:53.911299 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.911158    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9dbca512-ea16-4154-8a79-922a4780be51" (UID: "9dbca512-ea16-4154-8a79-922a4780be51"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:53.911455 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.911400    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-home\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:53.911455 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.911420    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-tmp-dir\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:53.911455 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.911422    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-model-cache" (OuterVolumeSpecName: "model-cache") pod "9dbca512-ea16-4154-8a79-922a4780be51" (UID: "9dbca512-ea16-4154-8a79-922a4780be51"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:53.913262 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.913235    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-dshm" (OuterVolumeSpecName: "dshm") pod "9dbca512-ea16-4154-8a79-922a4780be51" (UID: "9dbca512-ea16-4154-8a79-922a4780be51"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:08:53.913262 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.913249    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dbca512-ea16-4154-8a79-922a4780be51-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9dbca512-ea16-4154-8a79-922a4780be51" (UID: "9dbca512-ea16-4154-8a79-922a4780be51"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:08:53.913433 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:53.913320    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dbca512-ea16-4154-8a79-922a4780be51-kube-api-access-jzq6t" (OuterVolumeSpecName: "kube-api-access-jzq6t") pod "9dbca512-ea16-4154-8a79-922a4780be51" (UID: "9dbca512-ea16-4154-8a79-922a4780be51"). InnerVolumeSpecName "kube-api-access-jzq6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:08:54.012770 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.012731    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-dshm\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:54.012770 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.012769    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzq6t\" (UniqueName: \"kubernetes.io/projected/9dbca512-ea16-4154-8a79-922a4780be51-kube-api-access-jzq6t\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:54.012993 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.012782    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9dbca512-ea16-4154-8a79-922a4780be51-model-cache\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:54.012993 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.012794    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9dbca512-ea16-4154-8a79-922a4780be51-tls-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:08:54.024735 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.024694    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48661d42-238f-47ec-8c27-3d8b8b0ff8db" path="/var/lib/kubelet/pods/48661d42-238f-47ec-8c27-3d8b8b0ff8db/volumes"
Sep 11 12:08:54.025177 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.025156    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa5c5396-81ea-4410-b5da-102c1281bc43" path="/var/lib/kubelet/pods/fa5c5396-81ea-4410-b5da-102c1281bc43/volumes"
Sep 11 12:08:54.156896 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.156795    2562 generic.go:356] "Generic (PLEG): container finished" podID="9dbca512-ea16-4154-8a79-922a4780be51" containerID="c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66" exitCode=0
Sep 11 12:08:54.156896 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.156846    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"
Sep 11 12:08:54.156896 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.156869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp" event={"ID":"9dbca512-ea16-4154-8a79-922a4780be51","Type":"ContainerDied","Data":"c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66"}
Sep 11 12:08:54.157136 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.156908    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp" event={"ID":"9dbca512-ea16-4154-8a79-922a4780be51","Type":"ContainerDied","Data":"c5f4fe9d0db90d50a8265ce64f0a8cb1a1824422e85f8c95b2e294dfbfabcf05"}
Sep 11 12:08:54.157136 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.156930    2562 scope.go:117] "RemoveContainer" containerID="c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66"
Sep 11 12:08:54.165373 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.165352    2562 scope.go:117] "RemoveContainer" containerID="c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66"
Sep 11 12:08:54.165651 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:08:54.165632    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66\": container with ID starting with c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66 not found: ID does not exist" containerID="c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66"
Sep 11 12:08:54.165700 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.165658    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66"} err="failed to get container status \"c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66\": rpc error: code = NotFound desc = could not find container \"c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66\": container with ID starting with c0658a819e53813b95a079718b7277a6d37173c8bb7204035fcadd37be22ff66 not found: ID does not exist"
Sep 11 12:08:54.174211 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.174187    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"]
Sep 11 12:08:54.178239 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:54.178215    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-34511f/test-model-34511f-kserve-7458d56f4f-fsssp"]
Sep 11 12:08:55.211201 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211166    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"]
Sep 11 12:08:55.211656 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211588    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4de37e83-ef51-48d3-b777-bb131b6275ba" containerName="maas-api"
Sep 11 12:08:55.211656 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211602    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4de37e83-ef51-48d3-b777-bb131b6275ba" containerName="maas-api"
Sep 11 12:08:55.211656 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211625    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48661d42-238f-47ec-8c27-3d8b8b0ff8db" containerName="maas-api"
Sep 11 12:08:55.211656 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211632    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="48661d42-238f-47ec-8c27-3d8b8b0ff8db" containerName="maas-api"
Sep 11 12:08:55.211656 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211650    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ccdcdd2-4b16-454c-b479-bc772c24e046" containerName="payload-pre-processing"
Sep 11 12:08:55.211656 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211656    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ccdcdd2-4b16-454c-b479-bc772c24e046" containerName="payload-pre-processing"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211664    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f46e2e54-9a98-4137-a590-93135f7f7dd8" containerName="maas-api"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211669    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f46e2e54-9a98-4137-a590-93135f7f7dd8" containerName="maas-api"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211684    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa5c5396-81ea-4410-b5da-102c1281bc43" containerName="payload-pre-processing"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211689    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa5c5396-81ea-4410-b5da-102c1281bc43" containerName="payload-pre-processing"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211696    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600181af-041a-421b-9a90-63b012780dac" containerName="payload-processing"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211701    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="600181af-041a-421b-9a90-63b012780dac" containerName="payload-processing"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211709    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dbca512-ea16-4154-8a79-922a4780be51" containerName="main"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211714    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dbca512-ea16-4154-8a79-922a4780be51" containerName="main"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211780    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4de37e83-ef51-48d3-b777-bb131b6275ba" containerName="maas-api"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211786    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="600181af-041a-421b-9a90-63b012780dac" containerName="payload-processing"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211793    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ccdcdd2-4b16-454c-b479-bc772c24e046" containerName="payload-pre-processing"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211799    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="48661d42-238f-47ec-8c27-3d8b8b0ff8db" containerName="maas-api"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211805    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dbca512-ea16-4154-8a79-922a4780be51" containerName="main"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211809    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f46e2e54-9a98-4137-a590-93135f7f7dd8" containerName="maas-api"
Sep 11 12:08:55.211845 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.211818    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa5c5396-81ea-4410-b5da-102c1281bc43" containerName="payload-pre-processing"
Sep 11 12:08:55.216496 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.216476    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.218558 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.218536    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-40847e\"/\"openshift-service-ca.crt\""
Sep 11 12:08:55.219215 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.219188    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-40847e\"/\"kube-root-ca.crt\""
Sep 11 12:08:55.219333 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.219197    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-40847e\"/\"noten-model-40847e-kserve-self-signed-certs\""
Sep 11 12:08:55.219333 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.219197    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-40847e\"/\"default-dockercfg-b785n\""
Sep 11 12:08:55.224035 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.224009    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"]
Sep 11 12:08:55.325214 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.325178    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-home\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.325405 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.325241    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-dshm\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.325405 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.325271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-model-cache\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.325490 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.325450    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83c1c7dd-0535-48cf-ab08-3626a553dc46-tls-certs\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.325527 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.325494    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-tmp-dir\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.325561 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.325533    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk96l\" (UniqueName: \"kubernetes.io/projected/83c1c7dd-0535-48cf-ab08-3626a553dc46-kube-api-access-sk96l\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.426245 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.426208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83c1c7dd-0535-48cf-ab08-3626a553dc46-tls-certs\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.426245 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.426255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-tmp-dir\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.426498 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.426284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk96l\" (UniqueName: \"kubernetes.io/projected/83c1c7dd-0535-48cf-ab08-3626a553dc46-kube-api-access-sk96l\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.426498 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.426344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-home\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.426498 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.426406    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-dshm\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.426498 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.426434    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-model-cache\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.426762 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.426737    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-tmp-dir\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.426762 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.426756    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-home\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.426875 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.426825    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-model-cache\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.428643 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.428622    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-dshm\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.428836 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.428813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83c1c7dd-0535-48cf-ab08-3626a553dc46-tls-certs\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.434935 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.434910    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk96l\" (UniqueName: \"kubernetes.io/projected/83c1c7dd-0535-48cf-ab08-3626a553dc46-kube-api-access-sk96l\") pod \"noten-model-40847e-kserve-855d56cb47-cvhlz\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") " pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.527556 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.527457    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:08:55.801916 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:55.801616    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"]
Sep 11 12:08:55.810204 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:08:55.810172    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83c1c7dd_0535_48cf_ab08_3626a553dc46.slice/crio-f458977a850a884d39b99b93dbadc459aba579146db04dcbf0fac13220ee92c2 WatchSource:0}: Error finding container f458977a850a884d39b99b93dbadc459aba579146db04dcbf0fac13220ee92c2: Status 404 returned error can't find the container with id f458977a850a884d39b99b93dbadc459aba579146db04dcbf0fac13220ee92c2
Sep 11 12:08:56.025276 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:56.025243    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dbca512-ea16-4154-8a79-922a4780be51" path="/var/lib/kubelet/pods/9dbca512-ea16-4154-8a79-922a4780be51/volumes"
Sep 11 12:08:56.170592 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:56.170499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz" event={"ID":"83c1c7dd-0535-48cf-ab08-3626a553dc46","Type":"ContainerStarted","Data":"3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9"}
Sep 11 12:08:56.170592 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:56.170550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz" event={"ID":"83c1c7dd-0535-48cf-ab08-3626a553dc46","Type":"ContainerStarted","Data":"f458977a850a884d39b99b93dbadc459aba579146db04dcbf0fac13220ee92c2"}
Sep 11 12:08:56.200028 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:08:56.199971    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz" podStartSLOduration=1.199955649 podStartE2EDuration="1.199955649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:56.19771723 +0000 UTC m=+1044.698437499" watchObservedRunningTime="2026-09-11 12:08:56.199955649 +0000 UTC m=+1044.700675917"
Sep 11 12:09:05.528062 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:05.528015    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:09:05.528599 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:05.528082    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:09:05.540885 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:05.540851    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:09:06.232827 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:06.232797    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:09:09.378055 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.378007    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"]
Sep 11 12:09:09.378580 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.378250    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5" podUID="2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4" containerName="maas-api" containerID="cri-o://1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474" gracePeriod=30
Sep 11 12:09:09.627864 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.627838    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:09:09.765774 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.765743    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-metrics-certs\") pod \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") "
Sep 11 12:09:09.765774 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.765779    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-maas-api-tls\") pod \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") "
Sep 11 12:09:09.766021 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.765833    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqp8z\" (UniqueName: \"kubernetes.io/projected/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-kube-api-access-nqp8z\") pod \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") "
Sep 11 12:09:09.766021 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.765989    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-openshift-service-ca\") pod \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\" (UID: \"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4\") "
Sep 11 12:09:09.766465 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.766418    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4" (UID: "2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:09:09.768015 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.767982    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4" (UID: "2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:09:09.768015 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.768002    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4" (UID: "2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:09:09.768136 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.768031    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-kube-api-access-nqp8z" (OuterVolumeSpecName: "kube-api-access-nqp8z") pod "2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4" (UID: "2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4"). InnerVolumeSpecName "kube-api-access-nqp8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:09:09.867629 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.867581    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:09:09.867629 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.867623    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:09:09.867629 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.867634    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:09:09.867629 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:09.867642    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqp8z\" (UniqueName: \"kubernetes.io/projected/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4-kube-api-access-nqp8z\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:09:10.243157 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:10.243123    2562 generic.go:356] "Generic (PLEG): container finished" podID="2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4" containerID="1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474" exitCode=0
Sep 11 12:09:10.243326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:10.243163    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"
Sep 11 12:09:10.243326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:10.243186    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5" event={"ID":"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4","Type":"ContainerDied","Data":"1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474"}
Sep 11 12:09:10.243326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:10.243233    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5" event={"ID":"2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4","Type":"ContainerDied","Data":"a7b7ab6dc648bcf8d29c611323fed46c5a16e14f3c4dc735d25c7c2d8585d044"}
Sep 11 12:09:10.243326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:10.243256    2562 scope.go:117] "RemoveContainer" containerID="1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474"
Sep 11 12:09:10.257430 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:10.252621    2562 scope.go:117] "RemoveContainer" containerID="1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474"
Sep 11 12:09:10.257430 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:09:10.253002    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474\": container with ID starting with 1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474 not found: ID does not exist" containerID="1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474"
Sep 11 12:09:10.257430 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:10.253035    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474"} err="failed to get container status \"1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474\": rpc error: code = NotFound desc = could not find container \"1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474\": container with ID starting with 1a78b651d51fc42ba8a4c1eae9a7e31a6bca7d584f383163f22881fba57bf474 not found: ID does not exist"
Sep 11 12:09:10.262189 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:10.262162    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"]
Sep 11 12:09:10.266572 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:10.266549    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e42660-5b5b55b9d4-qvrb5"]
Sep 11 12:09:12.026961 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:12.026926    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4" path="/var/lib/kubelet/pods/2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4/volumes"
Sep 11 12:09:14.795039 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.795004    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"]
Sep 11 12:09:14.795536 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.795505    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4" containerName="maas-api"
Sep 11 12:09:14.795536 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.795527    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4" containerName="maas-api"
Sep 11 12:09:14.795681 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.795663    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bd4b6b5-481a-4fdc-b0ac-4b52b7fe43f4" containerName="maas-api"
Sep 11 12:09:14.800277 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.800255    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:14.802283 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.802261    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-c000cf-serving-cert\""
Sep 11 12:09:14.809495 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.809471    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"]
Sep 11 12:09:14.913356 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.913288    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:14.913526 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.913409    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:14.913574 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.913522    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:14.913574 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:14.913562    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq2vh\" (UniqueName: \"kubernetes.io/projected/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-kube-api-access-zq2vh\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:15.014257 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.014220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:15.014257 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.014256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq2vh\" (UniqueName: \"kubernetes.io/projected/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-kube-api-access-zq2vh\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:15.014540 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.014285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:15.014540 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.014364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:15.015011 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.014984    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:15.016785 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.016758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:15.016890 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.016873    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:15.022267 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.022243    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq2vh\" (UniqueName: \"kubernetes.io/projected/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-kube-api-access-zq2vh\") pod \"maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:15.114978 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.114885    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:15.394152 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.394122    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"]
Sep 11 12:09:15.397175 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:09:15.397131    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1d4b6d_4828_4f8f_8b07_79d24dfb1f0d.slice/crio-0f5c6817b80c60587da577148d1d6b43aac95389840bb50dcdfc15527ee07b00 WatchSource:0}: Error finding container 0f5c6817b80c60587da577148d1d6b43aac95389840bb50dcdfc15527ee07b00: Status 404 returned error can't find the container with id 0f5c6817b80c60587da577148d1d6b43aac95389840bb50dcdfc15527ee07b00
Sep 11 12:09:15.398682 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:15.398661    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:09:16.276231 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:16.276195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75" event={"ID":"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d","Type":"ContainerStarted","Data":"2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1"}
Sep 11 12:09:16.276631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:16.276248    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75" event={"ID":"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d","Type":"ContainerStarted","Data":"0f5c6817b80c60587da577148d1d6b43aac95389840bb50dcdfc15527ee07b00"}
Sep 11 12:09:16.276631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:16.276276    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:16.293358 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:16.293291    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75" podStartSLOduration=1.860836344 podStartE2EDuration="2.293276568s" totalImagesPullingTime="432.440224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:16.290832325 +0000 UTC m=+1064.791552593" watchObservedRunningTime="2026-09-11 12:09:16.293276568 +0000 UTC m=+1064.793996836"
Sep 11 12:09:20.810755 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:20.810721    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"]
Sep 11 12:09:21.909729 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:21.909691    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"]
Sep 11 12:09:21.913415 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:21.913394    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:21.915957 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:21.915938    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-88968d-serving-cert\""
Sep 11 12:09:21.927868 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:21.927836    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"]
Sep 11 12:09:21.980372 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:21.980340    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:21.980593 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:21.980410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xtnv\" (UniqueName: \"kubernetes.io/projected/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-kube-api-access-9xtnv\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:21.980593 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:21.980474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:21.980593 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:21.980529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.081997 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.081948    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.082218 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.082006    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.082218 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.082054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.082218 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:09:22.082174    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-serving-cert: secret "maas-api-e2e-shared-a-w3-88968d-serving-cert" not found
Sep 11 12:09:22.082442 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:09:22.082240    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-maas-api-tls podName:79ca0d64-6f85-4f7a-ab68-65f6f313cc1d nodeName:}" failed. No retries permitted until 2026-09-11 12:09:22.582222923 +0000 UTC m=+1071.082943169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-maas-api-tls") pod "maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j" (UID: "79ca0d64-6f85-4f7a-ab68-65f6f313cc1d") : secret "maas-api-e2e-shared-a-w3-88968d-serving-cert" not found
Sep 11 12:09:22.082442 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.082283    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xtnv\" (UniqueName: \"kubernetes.io/projected/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-kube-api-access-9xtnv\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.082788 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.082770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.084740 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.084715    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.090700 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.090680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xtnv\" (UniqueName: \"kubernetes.io/projected/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-kube-api-access-9xtnv\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.287457 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.287429    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:09:22.586761 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.586663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.589026 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.588995    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.824181 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.824141    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:22.866340 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.866233    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"]
Sep 11 12:09:22.870634 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.870615    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:09:22.873921 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.873746    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-88968d-dockercfg-4x2wd\""
Sep 11 12:09:22.875922 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.874703    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-88968d\""
Sep 11 12:09:22.885236 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.883481    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"]
Sep 11 12:09:22.890420 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.890199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8\" (UID: \"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:09:22.890420 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.890347    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtcd8\" (UniqueName: \"kubernetes.io/projected/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-kube-api-access-gtcd8\") pod \"payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8\" (UID: \"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:09:22.991581 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.991546    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcd8\" (UniqueName: \"kubernetes.io/projected/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-kube-api-access-gtcd8\") pod \"payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8\" (UID: \"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:09:22.991964 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.991657    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8\" (UID: \"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:09:22.992447 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:22.992418    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8\" (UID: \"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:09:23.000203 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:23.000175    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtcd8\" (UniqueName: \"kubernetes.io/projected/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-kube-api-access-gtcd8\") pod \"payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8\" (UID: \"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:09:23.101599 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:23.101573    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"]
Sep 11 12:09:23.104332 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:09:23.104289    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ca0d64_6f85_4f7a_ab68_65f6f313cc1d.slice/crio-d01e811127d513cea440e0c7312b2d48b55baf6d439601b89f32e8502c6d31ae WatchSource:0}: Error finding container d01e811127d513cea440e0c7312b2d48b55baf6d439601b89f32e8502c6d31ae: Status 404 returned error can't find the container with id d01e811127d513cea440e0c7312b2d48b55baf6d439601b89f32e8502c6d31ae
Sep 11 12:09:23.187818 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:23.187786    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:09:23.312396 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:23.312364    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j" event={"ID":"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d","Type":"ContainerStarted","Data":"d01e811127d513cea440e0c7312b2d48b55baf6d439601b89f32e8502c6d31ae"}
Sep 11 12:09:23.464288 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:23.464210    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"]
Sep 11 12:09:23.467783 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:09:23.467756    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441f8ca9_ac63_4db4_8f2b_cbb71a1a1bb2.slice/crio-285ad4b5698939a2e2188f5087d73f6fbc324873dc83a706c8c9360ef4681ac8 WatchSource:0}: Error finding container 285ad4b5698939a2e2188f5087d73f6fbc324873dc83a706c8c9360ef4681ac8: Status 404 returned error can't find the container with id 285ad4b5698939a2e2188f5087d73f6fbc324873dc83a706c8c9360ef4681ac8
Sep 11 12:09:24.320018 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:24.319932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j" event={"ID":"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d","Type":"ContainerStarted","Data":"6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699"}
Sep 11 12:09:24.320468 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:24.320047    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:24.321799 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:24.321772    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8" event={"ID":"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2","Type":"ContainerStarted","Data":"fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1"}
Sep 11 12:09:24.321936 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:24.321819    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8" event={"ID":"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2","Type":"ContainerStarted","Data":"285ad4b5698939a2e2188f5087d73f6fbc324873dc83a706c8c9360ef4681ac8"}
Sep 11 12:09:24.321936 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:24.321930    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:09:24.339198 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:24.339145    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j" podStartSLOduration=2.925261549 podStartE2EDuration="3.339128867s" totalImagesPullingTime="413.867318ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:24.338072495 +0000 UTC m=+1072.838792763" watchObservedRunningTime="2026-09-11 12:09:24.339128867 +0000 UTC m=+1072.839849136"
Sep 11 12:09:24.358825 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:24.358764    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8" podStartSLOduration=2.358743874 podStartE2EDuration="2.358743874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:24.355890457 +0000 UTC m=+1072.856610727" watchObservedRunningTime="2026-09-11 12:09:24.358743874 +0000 UTC m=+1072.859464145"
Sep 11 12:09:30.332293 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:30.332262    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:09:35.329325 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:35.329273    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:09:35.830169 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:35.830130    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz" podUID="83c1c7dd-0535-48cf-ab08-3626a553dc46" containerName="main" containerID="cri-o://3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9" gracePeriod=45
Sep 11 12:09:36.081553 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.081487    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:09:36.220667 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.220631    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-dshm\") pod \"83c1c7dd-0535-48cf-ab08-3626a553dc46\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") "
Sep 11 12:09:36.220861 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.220698    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-home\") pod \"83c1c7dd-0535-48cf-ab08-3626a553dc46\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") "
Sep 11 12:09:36.220861 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.220730    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83c1c7dd-0535-48cf-ab08-3626a553dc46-tls-certs\") pod \"83c1c7dd-0535-48cf-ab08-3626a553dc46\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") "
Sep 11 12:09:36.220861 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.220858    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk96l\" (UniqueName: \"kubernetes.io/projected/83c1c7dd-0535-48cf-ab08-3626a553dc46-kube-api-access-sk96l\") pod \"83c1c7dd-0535-48cf-ab08-3626a553dc46\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") "
Sep 11 12:09:36.221032 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.220891    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-model-cache\") pod \"83c1c7dd-0535-48cf-ab08-3626a553dc46\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") "
Sep 11 12:09:36.221032 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.220953    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-tmp-dir\") pod \"83c1c7dd-0535-48cf-ab08-3626a553dc46\" (UID: \"83c1c7dd-0535-48cf-ab08-3626a553dc46\") "
Sep 11 12:09:36.221032 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.220968    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-home" (OuterVolumeSpecName: "home") pod "83c1c7dd-0535-48cf-ab08-3626a553dc46" (UID: "83c1c7dd-0535-48cf-ab08-3626a553dc46"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:36.221215 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.221189    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-model-cache" (OuterVolumeSpecName: "model-cache") pod "83c1c7dd-0535-48cf-ab08-3626a553dc46" (UID: "83c1c7dd-0535-48cf-ab08-3626a553dc46"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:36.221290 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.221217    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-home\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:09:36.221290 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.221271    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83c1c7dd-0535-48cf-ab08-3626a553dc46" (UID: "83c1c7dd-0535-48cf-ab08-3626a553dc46"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:36.223086 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.223055    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83c1c7dd-0535-48cf-ab08-3626a553dc46-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83c1c7dd-0535-48cf-ab08-3626a553dc46" (UID: "83c1c7dd-0535-48cf-ab08-3626a553dc46"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:09:36.223187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.223108    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83c1c7dd-0535-48cf-ab08-3626a553dc46-kube-api-access-sk96l" (OuterVolumeSpecName: "kube-api-access-sk96l") pod "83c1c7dd-0535-48cf-ab08-3626a553dc46" (UID: "83c1c7dd-0535-48cf-ab08-3626a553dc46"). InnerVolumeSpecName "kube-api-access-sk96l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:09:36.223187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.223133    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-dshm" (OuterVolumeSpecName: "dshm") pod "83c1c7dd-0535-48cf-ab08-3626a553dc46" (UID: "83c1c7dd-0535-48cf-ab08-3626a553dc46"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:36.322700 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.322666    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83c1c7dd-0535-48cf-ab08-3626a553dc46-tls-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:09:36.322700 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.322698    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk96l\" (UniqueName: \"kubernetes.io/projected/83c1c7dd-0535-48cf-ab08-3626a553dc46-kube-api-access-sk96l\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:09:36.322700 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.322708    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-model-cache\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:09:36.322940 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.322716    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-tmp-dir\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:09:36.322940 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.322725    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83c1c7dd-0535-48cf-ab08-3626a553dc46-dshm\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:09:36.383676 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.383578    2562 generic.go:356] "Generic (PLEG): container finished" podID="83c1c7dd-0535-48cf-ab08-3626a553dc46" containerID="3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9" exitCode=0
Sep 11 12:09:36.383676 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.383644    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"
Sep 11 12:09:36.384097 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.383669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz" event={"ID":"83c1c7dd-0535-48cf-ab08-3626a553dc46","Type":"ContainerDied","Data":"3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9"}
Sep 11 12:09:36.384097 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.383719    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz" event={"ID":"83c1c7dd-0535-48cf-ab08-3626a553dc46","Type":"ContainerDied","Data":"f458977a850a884d39b99b93dbadc459aba579146db04dcbf0fac13220ee92c2"}
Sep 11 12:09:36.384097 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.383736    2562 scope.go:117] "RemoveContainer" containerID="3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9"
Sep 11 12:09:36.393468 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.393448    2562 scope.go:117] "RemoveContainer" containerID="3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9"
Sep 11 12:09:36.393777 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:09:36.393757    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9\": container with ID starting with 3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9 not found: ID does not exist" containerID="3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9"
Sep 11 12:09:36.393827 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.393785    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9"} err="failed to get container status \"3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9\": rpc error: code = NotFound desc = could not find container \"3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9\": container with ID starting with 3a2bbf641c57c7264ad469d0b41c99f82b87a647136f5d8bb2d4afed8fbcf7a9 not found: ID does not exist"
Sep 11 12:09:36.409885 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.409836    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"]
Sep 11 12:09:36.415118 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:36.415094    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-40847e/noten-model-40847e-kserve-855d56cb47-cvhlz"]
Sep 11 12:09:38.025983 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:38.025943    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83c1c7dd-0535-48cf-ab08-3626a553dc46" path="/var/lib/kubelet/pods/83c1c7dd-0535-48cf-ab08-3626a553dc46/volumes"
Sep 11 12:09:43.146548 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.146501    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"]
Sep 11 12:09:43.147102 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.147082    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c1c7dd-0535-48cf-ab08-3626a553dc46" containerName="main"
Sep 11 12:09:43.147185 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.147105    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c1c7dd-0535-48cf-ab08-3626a553dc46" containerName="main"
Sep 11 12:09:43.147243 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.147231    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="83c1c7dd-0535-48cf-ab08-3626a553dc46" containerName="main"
Sep 11 12:09:43.150523 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.150503    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.152836 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.152817    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-50e434-serving-cert\""
Sep 11 12:09:43.160279 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.160255    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"]
Sep 11 12:09:43.288411 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.288370    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.288614 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.288443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.288614 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.288535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4pg7\" (UniqueName: \"kubernetes.io/projected/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-kube-api-access-z4pg7\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.288614 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.288565    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.389538 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.389502    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.389747 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.389570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.389747 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.389636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4pg7\" (UniqueName: \"kubernetes.io/projected/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-kube-api-access-z4pg7\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.389747 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.389661    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.389747 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:09:43.389663    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-serving-cert: secret "maas-api-e2e-shared-b-w2-50e434-serving-cert" not found
Sep 11 12:09:43.389985 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:09:43.389752    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-maas-api-tls podName:a759c2a0-e03a-4925-9f75-24ef46ed5ab4 nodeName:}" failed. No retries permitted until 2026-09-11 12:09:43.88972964 +0000 UTC m=+1092.390449902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-maas-api-tls") pod "maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk" (UID: "a759c2a0-e03a-4925-9f75-24ef46ed5ab4") : secret "maas-api-e2e-shared-b-w2-50e434-serving-cert" not found
Sep 11 12:09:43.390371 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.390349    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.392036 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.392016    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.398033 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.397977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4pg7\" (UniqueName: \"kubernetes.io/projected/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-kube-api-access-z4pg7\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.896189 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.896155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:43.898655 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:43.898630    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:44.061478 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:44.061443    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:44.331252 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:44.331227    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"]
Sep 11 12:09:44.333718 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:09:44.333693    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda759c2a0_e03a_4925_9f75_24ef46ed5ab4.slice/crio-4b28ac5dd0bea6351d121a503ed407db10a8dbbd1312bc62d8986ac3ae464c1a WatchSource:0}: Error finding container 4b28ac5dd0bea6351d121a503ed407db10a8dbbd1312bc62d8986ac3ae464c1a: Status 404 returned error can't find the container with id 4b28ac5dd0bea6351d121a503ed407db10a8dbbd1312bc62d8986ac3ae464c1a
Sep 11 12:09:44.425275 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:44.425239    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk" event={"ID":"a759c2a0-e03a-4925-9f75-24ef46ed5ab4","Type":"ContainerStarted","Data":"4b28ac5dd0bea6351d121a503ed407db10a8dbbd1312bc62d8986ac3ae464c1a"}
Sep 11 12:09:45.432348 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:45.432291    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk" event={"ID":"a759c2a0-e03a-4925-9f75-24ef46ed5ab4","Type":"ContainerStarted","Data":"c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55"}
Sep 11 12:09:45.432793 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:45.432429    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:45.453825 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:45.453767    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk" podStartSLOduration=2.1824044049999998 podStartE2EDuration="2.45374872s" totalImagesPullingTime="271.344315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:45.452372101 +0000 UTC m=+1093.953092370" watchObservedRunningTime="2026-09-11 12:09:45.45374872 +0000 UTC m=+1093.954469016"
Sep 11 12:09:51.442381 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:51.442345    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:09:56.262528 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.262488    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"]
Sep 11 12:09:56.266437 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.266418    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.269491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.269463    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-4d3f83-serving-cert\""
Sep 11 12:09:56.288194 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.288163    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"]
Sep 11 12:09:56.317424 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.317389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwn7b\" (UniqueName: \"kubernetes.io/projected/44d41d00-f9b6-46b7-bff9-b8cecf425b75-kube-api-access-dwn7b\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.317612 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.317429    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.317612 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.317460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.317732 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.317652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d41d00-f9b6-46b7-bff9-b8cecf425b75-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.418370 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.418297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwn7b\" (UniqueName: \"kubernetes.io/projected/44d41d00-f9b6-46b7-bff9-b8cecf425b75-kube-api-access-dwn7b\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.418572 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.418379    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.418572 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.418537    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.418689 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.418662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d41d00-f9b6-46b7-bff9-b8cecf425b75-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.419266 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.419242    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d41d00-f9b6-46b7-bff9-b8cecf425b75-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.420793 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.420767    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.420959 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.420936    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.427704 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.427680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwn7b\" (UniqueName: \"kubernetes.io/projected/44d41d00-f9b6-46b7-bff9-b8cecf425b75-kube-api-access-dwn7b\") pod \"maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.578446 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.578353    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:56.856912 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.856833    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"]
Sep 11 12:09:56.860495 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:09:56.860453    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d41d00_f9b6_46b7_bff9_b8cecf425b75.slice/crio-c9e147511c6fff3313ccdc36e0f32079c78472b7c37d3970dc290833395eafce WatchSource:0}: Error finding container c9e147511c6fff3313ccdc36e0f32079c78472b7c37d3970dc290833395eafce: Status 404 returned error can't find the container with id c9e147511c6fff3313ccdc36e0f32079c78472b7c37d3970dc290833395eafce
Sep 11 12:09:56.960875 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.960843    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"]
Sep 11 12:09:56.965787 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.965770    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:09:56.967849 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.967825    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-4d3f83\""
Sep 11 12:09:56.967937 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.967904    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-4d3f83-dockercfg-m5rhr\""
Sep 11 12:09:56.975986 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:56.975961    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"]
Sep 11 12:09:57.024125 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.024094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n48zl\" (UniqueName: \"kubernetes.io/projected/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-kube-api-access-n48zl\") pod \"payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5\" (UID: \"2a108bfc-713f-4d3c-8c02-64ed9cf0031e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:09:57.024319 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.024143    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5\" (UID: \"2a108bfc-713f-4d3c-8c02-64ed9cf0031e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:09:57.125447 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.125355    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5\" (UID: \"2a108bfc-713f-4d3c-8c02-64ed9cf0031e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:09:57.125602 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.125487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n48zl\" (UniqueName: \"kubernetes.io/projected/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-kube-api-access-n48zl\") pod \"payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5\" (UID: \"2a108bfc-713f-4d3c-8c02-64ed9cf0031e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:09:57.126062 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.126041    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5\" (UID: \"2a108bfc-713f-4d3c-8c02-64ed9cf0031e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:09:57.136134 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.136105    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n48zl\" (UniqueName: \"kubernetes.io/projected/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-kube-api-access-n48zl\") pod \"payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5\" (UID: \"2a108bfc-713f-4d3c-8c02-64ed9cf0031e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:09:57.278144 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.278120    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:09:57.498247 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.498208    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx" event={"ID":"44d41d00-f9b6-46b7-bff9-b8cecf425b75","Type":"ContainerStarted","Data":"bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4"}
Sep 11 12:09:57.498494 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.498275    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx" event={"ID":"44d41d00-f9b6-46b7-bff9-b8cecf425b75","Type":"ContainerStarted","Data":"c9e147511c6fff3313ccdc36e0f32079c78472b7c37d3970dc290833395eafce"}
Sep 11 12:09:57.498494 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.498338    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:09:57.515626 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.515574    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx" podStartSLOduration=1.163499735 podStartE2EDuration="1.515557955s" totalImagesPullingTime="352.05822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:57.513673874 +0000 UTC m=+1106.014394142" watchObservedRunningTime="2026-09-11 12:09:57.515557955 +0000 UTC m=+1106.016278223"
Sep 11 12:09:57.551557 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:57.551530    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"]
Sep 11 12:09:57.553693 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:09:57.553651    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a108bfc_713f_4d3c_8c02_64ed9cf0031e.slice/crio-b2c49b115601fb01d3d49ef7eb9820a1ac06276afed5c40ec425ecc1dbe9ea0b WatchSource:0}: Error finding container b2c49b115601fb01d3d49ef7eb9820a1ac06276afed5c40ec425ecc1dbe9ea0b: Status 404 returned error can't find the container with id b2c49b115601fb01d3d49ef7eb9820a1ac06276afed5c40ec425ecc1dbe9ea0b
Sep 11 12:09:58.512403 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:58.512348    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5" event={"ID":"2a108bfc-713f-4d3c-8c02-64ed9cf0031e","Type":"ContainerStarted","Data":"6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f"}
Sep 11 12:09:58.512890 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:58.512468    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5" event={"ID":"2a108bfc-713f-4d3c-8c02-64ed9cf0031e","Type":"ContainerStarted","Data":"b2c49b115601fb01d3d49ef7eb9820a1ac06276afed5c40ec425ecc1dbe9ea0b"}
Sep 11 12:09:58.512890 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:58.512745    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:09:58.529662 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:09:58.529612    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5" podStartSLOduration=2.529599364 podStartE2EDuration="2.529599364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:58.527335366 +0000 UTC m=+1107.028055631" watchObservedRunningTime="2026-09-11 12:09:58.529599364 +0000 UTC m=+1107.030319633"
Sep 11 12:10:03.518275 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:03.518246    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:10:09.519833 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:09.519804    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:10:21.377694 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.377654    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"]
Sep 11 12:10:21.382325 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.382276    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.384617 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.384596    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fde9d154-serving-cert\""
Sep 11 12:10:21.393479 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.393450    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"]
Sep 11 12:10:21.467958 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.467917    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcb74\" (UniqueName: \"kubernetes.io/projected/54d72486-98f0-4649-b79d-3e470b5749c0-kube-api-access-jcb74\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.468166 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.467970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-metrics-certs\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.468166 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.467996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-maas-api-tls\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.468275 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.468167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54d72486-98f0-4649-b79d-3e470b5749c0-openshift-service-ca\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.569091 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.569055    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-maas-api-tls\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.569273 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.569152    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54d72486-98f0-4649-b79d-3e470b5749c0-openshift-service-ca\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.569273 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.569181    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcb74\" (UniqueName: \"kubernetes.io/projected/54d72486-98f0-4649-b79d-3e470b5749c0-kube-api-access-jcb74\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.569273 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.569204    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-metrics-certs\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.569273 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:10:21.569214    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-serving-cert: secret "maas-api-e2e-mt-fde9d154-serving-cert" not found
Sep 11 12:10:21.569497 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:10:21.569332    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-maas-api-tls podName:54d72486-98f0-4649-b79d-3e470b5749c0 nodeName:}" failed. No retries permitted until 2026-09-11 12:10:22.069286499 +0000 UTC m=+1130.570006748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-maas-api-tls") pod "maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4" (UID: "54d72486-98f0-4649-b79d-3e470b5749c0") : secret "maas-api-e2e-mt-fde9d154-serving-cert" not found
Sep 11 12:10:21.569828 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.569809    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54d72486-98f0-4649-b79d-3e470b5749c0-openshift-service-ca\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.571824 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.571798    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-metrics-certs\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:21.587808 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:21.587780    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcb74\" (UniqueName: \"kubernetes.io/projected/54d72486-98f0-4649-b79d-3e470b5749c0-kube-api-access-jcb74\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:22.075890 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:22.075831    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-maas-api-tls\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:22.078258 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:22.078231    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-maas-api-tls\") pod \"maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:22.295370 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:22.295329    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:22.572590 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:22.572561    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"]
Sep 11 12:10:22.575015 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:10:22.574987    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d72486_98f0_4649_b79d_3e470b5749c0.slice/crio-70589823cf600b952c202ec23d97db8db611717d9acbda5d9826a7020f21b82b WatchSource:0}: Error finding container 70589823cf600b952c202ec23d97db8db611717d9acbda5d9826a7020f21b82b: Status 404 returned error can't find the container with id 70589823cf600b952c202ec23d97db8db611717d9acbda5d9826a7020f21b82b
Sep 11 12:10:22.632878 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:22.632785    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4" event={"ID":"54d72486-98f0-4649-b79d-3e470b5749c0","Type":"ContainerStarted","Data":"70589823cf600b952c202ec23d97db8db611717d9acbda5d9826a7020f21b82b"}
Sep 11 12:10:23.639221 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:23.639180    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4" event={"ID":"54d72486-98f0-4649-b79d-3e470b5749c0","Type":"ContainerStarted","Data":"d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b"}
Sep 11 12:10:23.639635 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:23.639314    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:10:23.655938 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:23.655887    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4" podStartSLOduration=2.237414856 podStartE2EDuration="2.655871427s" totalImagesPullingTime="418.456571ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:23.654252386 +0000 UTC m=+1132.154972655" watchObservedRunningTime="2026-09-11 12:10:23.655871427 +0000 UTC m=+1132.156591694"
Sep 11 12:10:29.649992 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:10:29.649956    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:12:00.437663 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.437628    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"]
Sep 11 12:12:00.440233 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.437881    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4" podUID="54d72486-98f0-4649-b79d-3e470b5749c0" containerName="maas-api" containerID="cri-o://d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b" gracePeriod=30
Sep 11 12:12:00.697103 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.697009    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:12:00.763594 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.763563    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-metrics-certs\") pod \"54d72486-98f0-4649-b79d-3e470b5749c0\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") "
Sep 11 12:12:00.763775 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.763633    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-maas-api-tls\") pod \"54d72486-98f0-4649-b79d-3e470b5749c0\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") "
Sep 11 12:12:00.763775 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.763662    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcb74\" (UniqueName: \"kubernetes.io/projected/54d72486-98f0-4649-b79d-3e470b5749c0-kube-api-access-jcb74\") pod \"54d72486-98f0-4649-b79d-3e470b5749c0\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") "
Sep 11 12:12:00.763775 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.763725    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54d72486-98f0-4649-b79d-3e470b5749c0-openshift-service-ca\") pod \"54d72486-98f0-4649-b79d-3e470b5749c0\" (UID: \"54d72486-98f0-4649-b79d-3e470b5749c0\") "
Sep 11 12:12:00.764198 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.764165    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54d72486-98f0-4649-b79d-3e470b5749c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "54d72486-98f0-4649-b79d-3e470b5749c0" (UID: "54d72486-98f0-4649-b79d-3e470b5749c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:12:00.766034 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.766009    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54d72486-98f0-4649-b79d-3e470b5749c0-kube-api-access-jcb74" (OuterVolumeSpecName: "kube-api-access-jcb74") pod "54d72486-98f0-4649-b79d-3e470b5749c0" (UID: "54d72486-98f0-4649-b79d-3e470b5749c0"). InnerVolumeSpecName "kube-api-access-jcb74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:12:00.766134 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.766006    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "54d72486-98f0-4649-b79d-3e470b5749c0" (UID: "54d72486-98f0-4649-b79d-3e470b5749c0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:12:00.766197 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.766171    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "54d72486-98f0-4649-b79d-3e470b5749c0" (UID: "54d72486-98f0-4649-b79d-3e470b5749c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:12:00.865583 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.865551    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:12:00.865583 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.865579    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54d72486-98f0-4649-b79d-3e470b5749c0-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:12:00.865583 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.865589    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcb74\" (UniqueName: \"kubernetes.io/projected/54d72486-98f0-4649-b79d-3e470b5749c0-kube-api-access-jcb74\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:12:00.865813 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:00.865600    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54d72486-98f0-4649-b79d-3e470b5749c0-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:12:01.141067 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:01.140983    2562 generic.go:356] "Generic (PLEG): container finished" podID="54d72486-98f0-4649-b79d-3e470b5749c0" containerID="d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b" exitCode=0
Sep 11 12:12:01.141067 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:01.141036    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"
Sep 11 12:12:01.141246 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:01.141072    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4" event={"ID":"54d72486-98f0-4649-b79d-3e470b5749c0","Type":"ContainerDied","Data":"d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b"}
Sep 11 12:12:01.141246 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:01.141137    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4" event={"ID":"54d72486-98f0-4649-b79d-3e470b5749c0","Type":"ContainerDied","Data":"70589823cf600b952c202ec23d97db8db611717d9acbda5d9826a7020f21b82b"}
Sep 11 12:12:01.141246 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:01.141168    2562 scope.go:117] "RemoveContainer" containerID="d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b"
Sep 11 12:12:01.150557 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:01.150540    2562 scope.go:117] "RemoveContainer" containerID="d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b"
Sep 11 12:12:01.150859 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:12:01.150841    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b\": container with ID starting with d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b not found: ID does not exist" containerID="d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b"
Sep 11 12:12:01.150911 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:01.150869    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b"} err="failed to get container status \"d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b\": rpc error: code = NotFound desc = could not find container \"d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b\": container with ID starting with d972c8220e426ad27cad660e51dcc3ac596960a36291f07e2d5c29c44eaaca5b not found: ID does not exist"
Sep 11 12:12:01.161101 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:01.161073    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"]
Sep 11 12:12:01.165094 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:01.165071    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fde9d154-6cc7f445f4-vqnp4"]
Sep 11 12:12:02.033128 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:12:02.033094    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54d72486-98f0-4649-b79d-3e470b5749c0" path="/var/lib/kubelet/pods/54d72486-98f0-4649-b79d-3e470b5749c0/volumes"
Sep 11 12:13:49.831367 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.831328    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"]
Sep 11 12:13:49.831945 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.831927    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d72486-98f0-4649-b79d-3e470b5749c0" containerName="maas-api"
Sep 11 12:13:49.831996 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.831947    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d72486-98f0-4649-b79d-3e470b5749c0" containerName="maas-api"
Sep 11 12:13:49.832032 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.832016    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="54d72486-98f0-4649-b79d-3e470b5749c0" containerName="maas-api"
Sep 11 12:13:49.835380 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.835350    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:49.837682 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.837612    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-482cb0-serving-cert\""
Sep 11 12:13:49.838413 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.838387    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"]
Sep 11 12:13:49.899045 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.899010    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-maas-api-tls\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:49.899045 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.899049    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60a4af6f-c609-4f6f-824c-1969933928b5-openshift-service-ca\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:49.899260 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.899068    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddjh2\" (UniqueName: \"kubernetes.io/projected/60a4af6f-c609-4f6f-824c-1969933928b5-kube-api-access-ddjh2\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:49.899260 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.899115    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-metrics-certs\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:49.999905 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.999857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-maas-api-tls\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:50.000129 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.999913    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60a4af6f-c609-4f6f-824c-1969933928b5-openshift-service-ca\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:50.000129 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.999943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddjh2\" (UniqueName: \"kubernetes.io/projected/60a4af6f-c609-4f6f-824c-1969933928b5-kube-api-access-ddjh2\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:50.000129 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:49.999991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-metrics-certs\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:50.000129 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:13:50.000087    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-serving-cert: secret "maas-api-e2e-api-a-482cb0-serving-cert" not found
Sep 11 12:13:50.000411 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:13:50.000161    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-maas-api-tls podName:60a4af6f-c609-4f6f-824c-1969933928b5 nodeName:}" failed. No retries permitted until 2026-09-11 12:13:50.500143657 +0000 UTC m=+1339.000863925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-maas-api-tls") pod "maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4" (UID: "60a4af6f-c609-4f6f-824c-1969933928b5") : secret "maas-api-e2e-api-a-482cb0-serving-cert" not found
Sep 11 12:13:50.000556 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:50.000536    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60a4af6f-c609-4f6f-824c-1969933928b5-openshift-service-ca\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:50.002484 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:50.002462    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-metrics-certs\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:50.008652 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:50.008634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddjh2\" (UniqueName: \"kubernetes.io/projected/60a4af6f-c609-4f6f-824c-1969933928b5-kube-api-access-ddjh2\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:50.504839 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:50.504786    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-maas-api-tls\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:50.507391 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:50.507358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-maas-api-tls\") pod \"maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:50.747780 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:50.747746    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:51.020487 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:51.020457    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"]
Sep 11 12:13:51.022901 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:13:51.022871    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a4af6f_c609_4f6f_824c_1969933928b5.slice/crio-238a041e066bcf30b28ebd3af5ef86f6119b37595b14eceed24aa7e27714baf5 WatchSource:0}: Error finding container 238a041e066bcf30b28ebd3af5ef86f6119b37595b14eceed24aa7e27714baf5: Status 404 returned error can't find the container with id 238a041e066bcf30b28ebd3af5ef86f6119b37595b14eceed24aa7e27714baf5
Sep 11 12:13:51.714463 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:51.714408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4" event={"ID":"60a4af6f-c609-4f6f-824c-1969933928b5","Type":"ContainerStarted","Data":"1d1ca6ae16a849b21b151ea0898e9a62ce7d8c698268ee5c777b736f37420d6c"}
Sep 11 12:13:51.714633 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:51.714524    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:13:51.714633 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:51.714549    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4" event={"ID":"60a4af6f-c609-4f6f-824c-1969933928b5","Type":"ContainerStarted","Data":"238a041e066bcf30b28ebd3af5ef86f6119b37595b14eceed24aa7e27714baf5"}
Sep 11 12:13:51.734284 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:51.734231    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4" podStartSLOduration=2.189061608 podStartE2EDuration="2.734215449s" totalImagesPullingTime="545.153841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:13:51.732167328 +0000 UTC m=+1340.232887595" watchObservedRunningTime="2026-09-11 12:13:51.734215449 +0000 UTC m=+1340.234935717"
Sep 11 12:13:57.725499 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:13:57.725419    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:14:18.878629 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:18.878585    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"]
Sep 11 12:14:18.882318 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:18.882289    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:18.884340 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:18.884319    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-571fee-serving-cert\""
Sep 11 12:14:18.893186 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:18.893160    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"]
Sep 11 12:14:18.991530 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:18.991492    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss9d7\" (UniqueName: \"kubernetes.io/projected/03afa0d6-f89e-470a-a1fd-6076e2a0394b-kube-api-access-ss9d7\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:18.991530 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:18.991535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-metrics-certs\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:18.991738 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:18.991655    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-maas-api-tls\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:18.991738 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:18.991720    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03afa0d6-f89e-470a-a1fd-6076e2a0394b-openshift-service-ca\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:19.093187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.093145    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss9d7\" (UniqueName: \"kubernetes.io/projected/03afa0d6-f89e-470a-a1fd-6076e2a0394b-kube-api-access-ss9d7\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:19.093187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.093187    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-metrics-certs\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:19.093435 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.093237    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-maas-api-tls\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:19.093435 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.093275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03afa0d6-f89e-470a-a1fd-6076e2a0394b-openshift-service-ca\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:19.094007 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.093982    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03afa0d6-f89e-470a-a1fd-6076e2a0394b-openshift-service-ca\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:19.095749 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.095725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-metrics-certs\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:19.095892 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.095872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-maas-api-tls\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:19.101796 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.101768    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss9d7\" (UniqueName: \"kubernetes.io/projected/03afa0d6-f89e-470a-a1fd-6076e2a0394b-kube-api-access-ss9d7\") pod \"maas-api-e2e-api-b-571fee-5fc9df447c-mw27x\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:19.193753 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.193715    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:19.490530 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.490453    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"]
Sep 11 12:14:19.493829 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:14:19.493802    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03afa0d6_f89e_470a_a1fd_6076e2a0394b.slice/crio-d510f114351437656b1ff104af21966360a461c2adf97c50165009bb5a70539d WatchSource:0}: Error finding container d510f114351437656b1ff104af21966360a461c2adf97c50165009bb5a70539d: Status 404 returned error can't find the container with id d510f114351437656b1ff104af21966360a461c2adf97c50165009bb5a70539d
Sep 11 12:14:19.495431 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.495409    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:14:19.631326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.631271    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"]
Sep 11 12:14:19.635982 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.635965    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:14:19.639228 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.639202    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-571fee-dockercfg-4gd8t\""
Sep 11 12:14:19.640088 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.640063    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-571fee\""
Sep 11 12:14:19.641836 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.641812    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"]
Sep 11 12:14:19.646081 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.646054    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"]
Sep 11 12:14:19.646192 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.646167    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:14:19.658171 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.658141    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"]
Sep 11 12:14:19.699837 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.699809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5jtc\" (UniqueName: \"kubernetes.io/projected/3a375255-7402-49e2-888d-321ae57f85e6-kube-api-access-c5jtc\") pod \"payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp\" (UID: \"3a375255-7402-49e2-888d-321ae57f85e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:14:19.699975 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.699875    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5\" (UID: \"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9\") " pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:14:19.699975 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.699929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a375255-7402-49e2-888d-321ae57f85e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp\" (UID: \"3a375255-7402-49e2-888d-321ae57f85e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:14:19.700052 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.699989    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c7gm\" (UniqueName: \"kubernetes.io/projected/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-kube-api-access-6c7gm\") pod \"payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5\" (UID: \"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9\") " pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:14:19.800595 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.800505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5\" (UID: \"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9\") " pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:14:19.800762 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.800663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a375255-7402-49e2-888d-321ae57f85e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp\" (UID: \"3a375255-7402-49e2-888d-321ae57f85e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:14:19.800827 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.800762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c7gm\" (UniqueName: \"kubernetes.io/projected/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-kube-api-access-6c7gm\") pod \"payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5\" (UID: \"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9\") " pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:14:19.800934 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.800912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5jtc\" (UniqueName: \"kubernetes.io/projected/3a375255-7402-49e2-888d-321ae57f85e6-kube-api-access-c5jtc\") pod \"payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp\" (UID: \"3a375255-7402-49e2-888d-321ae57f85e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:14:19.801400 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.801375    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5\" (UID: \"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9\") " pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:14:19.801629 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.801609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a375255-7402-49e2-888d-321ae57f85e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp\" (UID: \"3a375255-7402-49e2-888d-321ae57f85e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:14:19.808847 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.808819    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c7gm\" (UniqueName: \"kubernetes.io/projected/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-kube-api-access-6c7gm\") pod \"payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5\" (UID: \"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9\") " pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:14:19.808992 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.808927    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5jtc\" (UniqueName: \"kubernetes.io/projected/3a375255-7402-49e2-888d-321ae57f85e6-kube-api-access-c5jtc\") pod \"payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp\" (UID: \"3a375255-7402-49e2-888d-321ae57f85e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:14:19.859058 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.859021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x" event={"ID":"03afa0d6-f89e-470a-a1fd-6076e2a0394b","Type":"ContainerStarted","Data":"d510f114351437656b1ff104af21966360a461c2adf97c50165009bb5a70539d"}
Sep 11 12:14:19.949828 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.949800    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:14:19.957893 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:19.957871    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:14:20.232187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.232144    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"]
Sep 11 12:14:20.245094 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.245065    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"]
Sep 11 12:14:20.252933 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:14:20.252904    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded62ff1f_6629_4f4f_b882_ca9c1773f8a9.slice/crio-55e4706f59032e93ffb41570e71dbcb6493f7058f984b37f0fe38ac8835e1c1e WatchSource:0}: Error finding container 55e4706f59032e93ffb41570e71dbcb6493f7058f984b37f0fe38ac8835e1c1e: Status 404 returned error can't find the container with id 55e4706f59032e93ffb41570e71dbcb6493f7058f984b37f0fe38ac8835e1c1e
Sep 11 12:14:20.867195 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.867158    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x" event={"ID":"03afa0d6-f89e-470a-a1fd-6076e2a0394b","Type":"ContainerStarted","Data":"5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561"}
Sep 11 12:14:20.867431 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.867221    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:20.868821 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.868794    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5" event={"ID":"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9","Type":"ContainerStarted","Data":"125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c"}
Sep 11 12:14:20.868921 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.868839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5" event={"ID":"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9","Type":"ContainerStarted","Data":"55e4706f59032e93ffb41570e71dbcb6493f7058f984b37f0fe38ac8835e1c1e"}
Sep 11 12:14:20.868921 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.868865    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:14:20.870572 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.870544    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp" event={"ID":"3a375255-7402-49e2-888d-321ae57f85e6","Type":"ContainerStarted","Data":"17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847"}
Sep 11 12:14:20.870674 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.870595    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp" event={"ID":"3a375255-7402-49e2-888d-321ae57f85e6","Type":"ContainerStarted","Data":"f9268bdc7c44ce18c27cd03d9f81863a26ed15bfd9b21457e11641df9e47fb9e"}
Sep 11 12:14:20.870674 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.870625    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:14:20.883956 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.883898    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x" podStartSLOduration=2.481557576 podStartE2EDuration="2.883883382s" totalImagesPullingTime="402.325806ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:14:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:14:20.881526505 +0000 UTC m=+1369.382246777" watchObservedRunningTime="2026-09-11 12:14:20.883883382 +0000 UTC m=+1369.384603665"
Sep 11 12:14:20.897582 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.897526    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp" podStartSLOduration=1.8975087240000001 podStartE2EDuration="1.897508724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:14:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:14:20.896089105 +0000 UTC m=+1369.396809367" watchObservedRunningTime="2026-09-11 12:14:20.897508724 +0000 UTC m=+1369.398228992"
Sep 11 12:14:20.910836 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:20.910787    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5" podStartSLOduration=1.9107701160000001 podStartE2EDuration="1.910770116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:14:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:14:20.90954704 +0000 UTC m=+1369.410267312" watchObservedRunningTime="2026-09-11 12:14:20.910770116 +0000 UTC m=+1369.411490383"
Sep 11 12:14:26.888858 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:26.888830    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:14:31.885168 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:31.885133    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:14:31.885627 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:14:31.885555    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:15:00.917368 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:00.917333    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"]
Sep 11 12:15:00.918037 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:00.917528    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4" podUID="60a4af6f-c609-4f6f-824c-1969933928b5" containerName="maas-api" containerID="cri-o://1d1ca6ae16a849b21b151ea0898e9a62ce7d8c698268ee5c777b736f37420d6c" gracePeriod=30
Sep 11 12:15:01.101956 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.101922    2562 generic.go:356] "Generic (PLEG): container finished" podID="60a4af6f-c609-4f6f-824c-1969933928b5" containerID="1d1ca6ae16a849b21b151ea0898e9a62ce7d8c698268ee5c777b736f37420d6c" exitCode=0
Sep 11 12:15:01.102114 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.101994    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4" event={"ID":"60a4af6f-c609-4f6f-824c-1969933928b5","Type":"ContainerDied","Data":"1d1ca6ae16a849b21b151ea0898e9a62ce7d8c698268ee5c777b736f37420d6c"}
Sep 11 12:15:01.180769 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.180728    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:15:01.212039 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.212005    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-metrics-certs\") pod \"60a4af6f-c609-4f6f-824c-1969933928b5\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") "
Sep 11 12:15:01.212233 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.212103    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddjh2\" (UniqueName: \"kubernetes.io/projected/60a4af6f-c609-4f6f-824c-1969933928b5-kube-api-access-ddjh2\") pod \"60a4af6f-c609-4f6f-824c-1969933928b5\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") "
Sep 11 12:15:01.212233 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.212158    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-maas-api-tls\") pod \"60a4af6f-c609-4f6f-824c-1969933928b5\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") "
Sep 11 12:15:01.212233 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.212214    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60a4af6f-c609-4f6f-824c-1969933928b5-openshift-service-ca\") pod \"60a4af6f-c609-4f6f-824c-1969933928b5\" (UID: \"60a4af6f-c609-4f6f-824c-1969933928b5\") "
Sep 11 12:15:01.212691 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.212661    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60a4af6f-c609-4f6f-824c-1969933928b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60a4af6f-c609-4f6f-824c-1969933928b5" (UID: "60a4af6f-c609-4f6f-824c-1969933928b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:01.214405 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.214370    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60a4af6f-c609-4f6f-824c-1969933928b5" (UID: "60a4af6f-c609-4f6f-824c-1969933928b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:15:01.214700 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.214678    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a4af6f-c609-4f6f-824c-1969933928b5-kube-api-access-ddjh2" (OuterVolumeSpecName: "kube-api-access-ddjh2") pod "60a4af6f-c609-4f6f-824c-1969933928b5" (UID: "60a4af6f-c609-4f6f-824c-1969933928b5"). InnerVolumeSpecName "kube-api-access-ddjh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:01.214789 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.214675    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "60a4af6f-c609-4f6f-824c-1969933928b5" (UID: "60a4af6f-c609-4f6f-824c-1969933928b5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:15:01.314111 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.313930    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddjh2\" (UniqueName: \"kubernetes.io/projected/60a4af6f-c609-4f6f-824c-1969933928b5-kube-api-access-ddjh2\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:01.314282 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.314120    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:01.314282 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.314134    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60a4af6f-c609-4f6f-824c-1969933928b5-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:01.314282 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:01.314144    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a4af6f-c609-4f6f-824c-1969933928b5-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:02.108825 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:02.108785    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4" event={"ID":"60a4af6f-c609-4f6f-824c-1969933928b5","Type":"ContainerDied","Data":"238a041e066bcf30b28ebd3af5ef86f6119b37595b14eceed24aa7e27714baf5"}
Sep 11 12:15:02.109246 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:02.108850    2562 scope.go:117] "RemoveContainer" containerID="1d1ca6ae16a849b21b151ea0898e9a62ce7d8c698268ee5c777b736f37420d6c"
Sep 11 12:15:02.109246 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:02.108799    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"
Sep 11 12:15:02.124012 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:02.123988    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"]
Sep 11 12:15:02.127045 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:02.127023    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-482cb0-7585c87cfb-rcmw4"]
Sep 11 12:15:04.025404 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:04.025367    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60a4af6f-c609-4f6f-824c-1969933928b5" path="/var/lib/kubelet/pods/60a4af6f-c609-4f6f-824c-1969933928b5/volumes"
Sep 11 12:15:23.506479 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:23.506440    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"]
Sep 11 12:15:23.507632 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:23.506702    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x" podUID="03afa0d6-f89e-470a-a1fd-6076e2a0394b" containerName="maas-api" containerID="cri-o://5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561" gracePeriod=30
Sep 11 12:15:23.560472 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:23.560428    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"]
Sep 11 12:15:23.560751 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:23.560694    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5" podUID="ed62ff1f-6629-4f4f-b882-ca9c1773f8a9" containerName="payload-processing" containerID="cri-o://125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c" gracePeriod=30
Sep 11 12:15:23.571571 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:23.571533    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"]
Sep 11 12:15:23.571780 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:23.571754    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp" podUID="3a375255-7402-49e2-888d-321ae57f85e6" containerName="payload-pre-processing" containerID="cri-o://17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847" gracePeriod=30
Sep 11 12:15:23.877425 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:23.877395    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:15:23.881094 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:23.881075    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:15:23.885819 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:23.885801    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:15:24.043383 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043353    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a375255-7402-49e2-888d-321ae57f85e6-plugins-config-volume\") pod \"3a375255-7402-49e2-888d-321ae57f85e6\" (UID: \"3a375255-7402-49e2-888d-321ae57f85e6\") "
Sep 11 12:15:24.043538 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043396    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03afa0d6-f89e-470a-a1fd-6076e2a0394b-openshift-service-ca\") pod \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") "
Sep 11 12:15:24.043538 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043428    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-metrics-certs\") pod \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") "
Sep 11 12:15:24.043538 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043476    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6c7gm\" (UniqueName: \"kubernetes.io/projected/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-kube-api-access-6c7gm\") pod \"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9\" (UID: \"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9\") "
Sep 11 12:15:24.043538 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043533    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-maas-api-tls\") pod \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") "
Sep 11 12:15:24.043719 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043559    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-plugins-config-volume\") pod \"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9\" (UID: \"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9\") "
Sep 11 12:15:24.043719 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043582    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss9d7\" (UniqueName: \"kubernetes.io/projected/03afa0d6-f89e-470a-a1fd-6076e2a0394b-kube-api-access-ss9d7\") pod \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\" (UID: \"03afa0d6-f89e-470a-a1fd-6076e2a0394b\") "
Sep 11 12:15:24.043719 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043665    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5jtc\" (UniqueName: \"kubernetes.io/projected/3a375255-7402-49e2-888d-321ae57f85e6-kube-api-access-c5jtc\") pod \"3a375255-7402-49e2-888d-321ae57f85e6\" (UID: \"3a375255-7402-49e2-888d-321ae57f85e6\") "
Sep 11 12:15:24.043909 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043841    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03afa0d6-f89e-470a-a1fd-6076e2a0394b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "03afa0d6-f89e-470a-a1fd-6076e2a0394b" (UID: "03afa0d6-f89e-470a-a1fd-6076e2a0394b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:24.043975 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043899    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a375255-7402-49e2-888d-321ae57f85e6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3a375255-7402-49e2-888d-321ae57f85e6" (UID: "3a375255-7402-49e2-888d-321ae57f85e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:24.043975 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.043947    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed62ff1f-6629-4f4f-b882-ca9c1773f8a9" (UID: "ed62ff1f-6629-4f4f-b882-ca9c1773f8a9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:24.044084 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.044027    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03afa0d6-f89e-470a-a1fd-6076e2a0394b-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:24.044084 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.044043    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:24.044084 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.044056    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a375255-7402-49e2-888d-321ae57f85e6-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:24.045809 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.045773    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03afa0d6-f89e-470a-a1fd-6076e2a0394b-kube-api-access-ss9d7" (OuterVolumeSpecName: "kube-api-access-ss9d7") pod "03afa0d6-f89e-470a-a1fd-6076e2a0394b" (UID: "03afa0d6-f89e-470a-a1fd-6076e2a0394b"). InnerVolumeSpecName "kube-api-access-ss9d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:24.045929 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.045855    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-kube-api-access-6c7gm" (OuterVolumeSpecName: "kube-api-access-6c7gm") pod "ed62ff1f-6629-4f4f-b882-ca9c1773f8a9" (UID: "ed62ff1f-6629-4f4f-b882-ca9c1773f8a9"). InnerVolumeSpecName "kube-api-access-6c7gm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:24.045929 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.045910    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a375255-7402-49e2-888d-321ae57f85e6-kube-api-access-c5jtc" (OuterVolumeSpecName: "kube-api-access-c5jtc") pod "3a375255-7402-49e2-888d-321ae57f85e6" (UID: "3a375255-7402-49e2-888d-321ae57f85e6"). InnerVolumeSpecName "kube-api-access-c5jtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:24.046032 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.046014    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "03afa0d6-f89e-470a-a1fd-6076e2a0394b" (UID: "03afa0d6-f89e-470a-a1fd-6076e2a0394b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:15:24.046074 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.046030    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "03afa0d6-f89e-470a-a1fd-6076e2a0394b" (UID: "03afa0d6-f89e-470a-a1fd-6076e2a0394b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:15:24.145647 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.145567    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:24.145647 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.145599    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6c7gm\" (UniqueName: \"kubernetes.io/projected/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9-kube-api-access-6c7gm\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:24.145647 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.145609    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03afa0d6-f89e-470a-a1fd-6076e2a0394b-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:24.145647 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.145619    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss9d7\" (UniqueName: \"kubernetes.io/projected/03afa0d6-f89e-470a-a1fd-6076e2a0394b-kube-api-access-ss9d7\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:24.145647 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.145628    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5jtc\" (UniqueName: \"kubernetes.io/projected/3a375255-7402-49e2-888d-321ae57f85e6-kube-api-access-c5jtc\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:24.233454 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.233417    2562 generic.go:356] "Generic (PLEG): container finished" podID="03afa0d6-f89e-470a-a1fd-6076e2a0394b" containerID="5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561" exitCode=0
Sep 11 12:15:24.233631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.233467    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"
Sep 11 12:15:24.233631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.233503    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x" event={"ID":"03afa0d6-f89e-470a-a1fd-6076e2a0394b","Type":"ContainerDied","Data":"5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561"}
Sep 11 12:15:24.233631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.233571    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x" event={"ID":"03afa0d6-f89e-470a-a1fd-6076e2a0394b","Type":"ContainerDied","Data":"d510f114351437656b1ff104af21966360a461c2adf97c50165009bb5a70539d"}
Sep 11 12:15:24.233631 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.233599    2562 scope.go:117] "RemoveContainer" containerID="5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561"
Sep 11 12:15:24.234896 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.234865    2562 generic.go:356] "Generic (PLEG): container finished" podID="ed62ff1f-6629-4f4f-b882-ca9c1773f8a9" containerID="125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c" exitCode=0
Sep 11 12:15:24.235012 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.234914    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"
Sep 11 12:15:24.235012 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.234913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5" event={"ID":"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9","Type":"ContainerDied","Data":"125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c"}
Sep 11 12:15:24.235012 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.234967    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5" event={"ID":"ed62ff1f-6629-4f4f-b882-ca9c1773f8a9","Type":"ContainerDied","Data":"55e4706f59032e93ffb41570e71dbcb6493f7058f984b37f0fe38ac8835e1c1e"}
Sep 11 12:15:24.236681 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.236657    2562 generic.go:356] "Generic (PLEG): container finished" podID="3a375255-7402-49e2-888d-321ae57f85e6" containerID="17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847" exitCode=0
Sep 11 12:15:24.236801 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.236707    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"
Sep 11 12:15:24.236801 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.236722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp" event={"ID":"3a375255-7402-49e2-888d-321ae57f85e6","Type":"ContainerDied","Data":"17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847"}
Sep 11 12:15:24.236801 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.236761    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp" event={"ID":"3a375255-7402-49e2-888d-321ae57f85e6","Type":"ContainerDied","Data":"f9268bdc7c44ce18c27cd03d9f81863a26ed15bfd9b21457e11641df9e47fb9e"}
Sep 11 12:15:24.245811 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.245775    2562 scope.go:117] "RemoveContainer" containerID="5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561"
Sep 11 12:15:24.246377 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:15:24.246350    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561\": container with ID starting with 5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561 not found: ID does not exist" containerID="5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561"
Sep 11 12:15:24.246472 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.246389    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561"} err="failed to get container status \"5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561\": rpc error: code = NotFound desc = could not find container \"5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561\": container with ID starting with 5220c78f79a1e0aa6367a4244de87101bf5e7bd32cf5cae7fd43e93988b21561 not found: ID does not exist"
Sep 11 12:15:24.246472 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.246411    2562 scope.go:117] "RemoveContainer" containerID="125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c"
Sep 11 12:15:24.255371 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.255354    2562 scope.go:117] "RemoveContainer" containerID="125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c"
Sep 11 12:15:24.255651 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:15:24.255628    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c\": container with ID starting with 125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c not found: ID does not exist" containerID="125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c"
Sep 11 12:15:24.255715 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.255656    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c"} err="failed to get container status \"125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c\": rpc error: code = NotFound desc = could not find container \"125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c\": container with ID starting with 125a8da291f28ecbcf381c6ec4dfaa0fd446ad05dcd32676564c2d339232cf0c not found: ID does not exist"
Sep 11 12:15:24.255715 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.255673    2562 scope.go:117] "RemoveContainer" containerID="17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847"
Sep 11 12:15:24.260620 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.260598    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"]
Sep 11 12:15:24.264161 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.264144    2562 scope.go:117] "RemoveContainer" containerID="17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847"
Sep 11 12:15:24.264469 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:15:24.264450    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847\": container with ID starting with 17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847 not found: ID does not exist" containerID="17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847"
Sep 11 12:15:24.264536 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.264477    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847"} err="failed to get container status \"17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847\": rpc error: code = NotFound desc = could not find container \"17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847\": container with ID starting with 17917b8e6d927e45d7276670426235819bd1e8650bb51dc7e4ba45bb96ae2847 not found: ID does not exist"
Sep 11 12:15:24.264536 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.264526    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-571fee-5fc9df447c-mw27x"]
Sep 11 12:15:24.272972 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.272951    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"]
Sep 11 12:15:24.277528 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.277506    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-571fee-74565cbc6b-wm7pp"]
Sep 11 12:15:24.286395 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.286371    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"]
Sep 11 12:15:24.292107 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:24.292086    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-571fee-77f7c54759-sv4s5"]
Sep 11 12:15:26.025376 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:26.025340    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03afa0d6-f89e-470a-a1fd-6076e2a0394b" path="/var/lib/kubelet/pods/03afa0d6-f89e-470a-a1fd-6076e2a0394b/volumes"
Sep 11 12:15:26.025760 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:26.025710    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a375255-7402-49e2-888d-321ae57f85e6" path="/var/lib/kubelet/pods/3a375255-7402-49e2-888d-321ae57f85e6/volumes"
Sep 11 12:15:26.026046 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:26.026033    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed62ff1f-6629-4f4f-b882-ca9c1773f8a9" path="/var/lib/kubelet/pods/ed62ff1f-6629-4f4f-b882-ca9c1773f8a9/volumes"
Sep 11 12:15:40.911272 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:40.911237    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"]
Sep 11 12:15:40.911871 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:40.911833    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75" podUID="9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d" containerName="maas-api" containerID="cri-o://2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1" gracePeriod=30
Sep 11 12:15:41.169112 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.169052    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:15:41.195205 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.195169    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-maas-api-tls\") pod \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") "
Sep 11 12:15:41.195369 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.195319    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-openshift-service-ca\") pod \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") "
Sep 11 12:15:41.195478 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.195400    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq2vh\" (UniqueName: \"kubernetes.io/projected/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-kube-api-access-zq2vh\") pod \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") "
Sep 11 12:15:41.195478 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.195447    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-metrics-certs\") pod \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\" (UID: \"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d\") "
Sep 11 12:15:41.195665 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.195634    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d" (UID: "9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:41.195770 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.195751    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:41.197602 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.197573    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d" (UID: "9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:15:41.197710 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.197612    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d" (UID: "9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:15:41.197710 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.197636    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-kube-api-access-zq2vh" (OuterVolumeSpecName: "kube-api-access-zq2vh") pod "9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d" (UID: "9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d"). InnerVolumeSpecName "kube-api-access-zq2vh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:41.296995 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.296962    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq2vh\" (UniqueName: \"kubernetes.io/projected/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-kube-api-access-zq2vh\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:41.296995 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.296993    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:41.296995 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.297002    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:41.335193 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.335156    2562 generic.go:356] "Generic (PLEG): container finished" podID="9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d" containerID="2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1" exitCode=0
Sep 11 12:15:41.335421 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.335229    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"
Sep 11 12:15:41.335421 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.335249    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75" event={"ID":"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d","Type":"ContainerDied","Data":"2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1"}
Sep 11 12:15:41.335421 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.335376    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75" event={"ID":"9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d","Type":"ContainerDied","Data":"0f5c6817b80c60587da577148d1d6b43aac95389840bb50dcdfc15527ee07b00"}
Sep 11 12:15:41.335421 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.335413    2562 scope.go:117] "RemoveContainer" containerID="2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1"
Sep 11 12:15:41.346596 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.346575    2562 scope.go:117] "RemoveContainer" containerID="2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1"
Sep 11 12:15:41.346928 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:15:41.346900    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1\": container with ID starting with 2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1 not found: ID does not exist" containerID="2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1"
Sep 11 12:15:41.347028 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.346929    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1"} err="failed to get container status \"2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1\": rpc error: code = NotFound desc = could not find container \"2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1\": container with ID starting with 2be6c04b90691649d79c38245d4acbc03a226bf4f46fc747b73c34dfc9ba31c1 not found: ID does not exist"
Sep 11 12:15:41.364521 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.364488    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"]
Sep 11 12:15:41.367950 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:41.367923    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c000cf-6b765dcfd4-p7p75"]
Sep 11 12:15:42.025425 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:42.025390    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d" path="/var/lib/kubelet/pods/9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d/volumes"
Sep 11 12:15:58.446120 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.446086    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"]
Sep 11 12:15:58.446594 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.446293    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk" podUID="a759c2a0-e03a-4925-9f75-24ef46ed5ab4" containerName="maas-api" containerID="cri-o://c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55" gracePeriod=30
Sep 11 12:15:58.698866 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.698790    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:15:58.769112 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.769073    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-maas-api-tls\") pod \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") "
Sep 11 12:15:58.769112 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.769113    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4pg7\" (UniqueName: \"kubernetes.io/projected/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-kube-api-access-z4pg7\") pod \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") "
Sep 11 12:15:58.769354 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.769163    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-metrics-certs\") pod \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") "
Sep 11 12:15:58.769354 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.769251    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-openshift-service-ca\") pod \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\" (UID: \"a759c2a0-e03a-4925-9f75-24ef46ed5ab4\") "
Sep 11 12:15:58.769670 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.769641    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a759c2a0-e03a-4925-9f75-24ef46ed5ab4" (UID: "a759c2a0-e03a-4925-9f75-24ef46ed5ab4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:15:58.771396 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.771366    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a759c2a0-e03a-4925-9f75-24ef46ed5ab4" (UID: "a759c2a0-e03a-4925-9f75-24ef46ed5ab4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:15:58.771521 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.771445    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-kube-api-access-z4pg7" (OuterVolumeSpecName: "kube-api-access-z4pg7") pod "a759c2a0-e03a-4925-9f75-24ef46ed5ab4" (UID: "a759c2a0-e03a-4925-9f75-24ef46ed5ab4"). InnerVolumeSpecName "kube-api-access-z4pg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:15:58.771585 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.771510    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a759c2a0-e03a-4925-9f75-24ef46ed5ab4" (UID: "a759c2a0-e03a-4925-9f75-24ef46ed5ab4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:15:58.870107 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.870063    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:58.870107 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.870098    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:58.870107 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.870114    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4pg7\" (UniqueName: \"kubernetes.io/projected/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-kube-api-access-z4pg7\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:58.870382 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:58.870126    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a759c2a0-e03a-4925-9f75-24ef46ed5ab4-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:15:59.431958 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:59.431926    2562 generic.go:356] "Generic (PLEG): container finished" podID="a759c2a0-e03a-4925-9f75-24ef46ed5ab4" containerID="c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55" exitCode=0
Sep 11 12:15:59.432128 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:59.431974    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"
Sep 11 12:15:59.432128 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:59.431985    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk" event={"ID":"a759c2a0-e03a-4925-9f75-24ef46ed5ab4","Type":"ContainerDied","Data":"c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55"}
Sep 11 12:15:59.432128 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:59.432029    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk" event={"ID":"a759c2a0-e03a-4925-9f75-24ef46ed5ab4","Type":"ContainerDied","Data":"4b28ac5dd0bea6351d121a503ed407db10a8dbbd1312bc62d8986ac3ae464c1a"}
Sep 11 12:15:59.432128 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:59.432050    2562 scope.go:117] "RemoveContainer" containerID="c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55"
Sep 11 12:15:59.441645 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:59.441623    2562 scope.go:117] "RemoveContainer" containerID="c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55"
Sep 11 12:15:59.441918 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:15:59.441898    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55\": container with ID starting with c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55 not found: ID does not exist" containerID="c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55"
Sep 11 12:15:59.441965 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:59.441927    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55"} err="failed to get container status \"c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55\": rpc error: code = NotFound desc = could not find container \"c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55\": container with ID starting with c39ef1ae921270303fe84534d58038761fec6553889a519875818cd3938e9a55 not found: ID does not exist"
Sep 11 12:15:59.452364 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:59.452339    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"]
Sep 11 12:15:59.458486 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:15:59.458463    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-50e434-798778665b-fbfpk"]
Sep 11 12:16:00.024739 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:00.024704    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a759c2a0-e03a-4925-9f75-24ef46ed5ab4" path="/var/lib/kubelet/pods/a759c2a0-e03a-4925-9f75-24ef46ed5ab4/volumes"
Sep 11 12:16:58.028646 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.028561    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"]
Sep 11 12:16:58.029745 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029718    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a4af6f-c609-4f6f-824c-1969933928b5" containerName="maas-api"
Sep 11 12:16:58.029745 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029745    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a4af6f-c609-4f6f-824c-1969933928b5" containerName="maas-api"
Sep 11 12:16:58.029926 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029761    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed62ff1f-6629-4f4f-b882-ca9c1773f8a9" containerName="payload-processing"
Sep 11 12:16:58.029926 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029770    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed62ff1f-6629-4f4f-b882-ca9c1773f8a9" containerName="payload-processing"
Sep 11 12:16:58.029926 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029792    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a759c2a0-e03a-4925-9f75-24ef46ed5ab4" containerName="maas-api"
Sep 11 12:16:58.029926 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029801    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a759c2a0-e03a-4925-9f75-24ef46ed5ab4" containerName="maas-api"
Sep 11 12:16:58.029926 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029814    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d" containerName="maas-api"
Sep 11 12:16:58.029926 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029822    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d" containerName="maas-api"
Sep 11 12:16:58.029926 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029837    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03afa0d6-f89e-470a-a1fd-6076e2a0394b" containerName="maas-api"
Sep 11 12:16:58.029926 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029845    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="03afa0d6-f89e-470a-a1fd-6076e2a0394b" containerName="maas-api"
Sep 11 12:16:58.029926 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029856    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a375255-7402-49e2-888d-321ae57f85e6" containerName="payload-pre-processing"
Sep 11 12:16:58.029926 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029865    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a375255-7402-49e2-888d-321ae57f85e6" containerName="payload-pre-processing"
Sep 11 12:16:58.030481 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029969    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1d4b6d-4828-4f8f-8b07-79d24dfb1f0d" containerName="maas-api"
Sep 11 12:16:58.030481 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029979    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a759c2a0-e03a-4925-9f75-24ef46ed5ab4" containerName="maas-api"
Sep 11 12:16:58.030481 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029991    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="03afa0d6-f89e-470a-a1fd-6076e2a0394b" containerName="maas-api"
Sep 11 12:16:58.030481 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.029999    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a375255-7402-49e2-888d-321ae57f85e6" containerName="payload-pre-processing"
Sep 11 12:16:58.030481 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.030009    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed62ff1f-6629-4f4f-b882-ca9c1773f8a9" containerName="payload-processing"
Sep 11 12:16:58.030481 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.030017    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="60a4af6f-c609-4f6f-824c-1969933928b5" containerName="maas-api"
Sep 11 12:16:58.034698 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.034678    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.037040 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.037017    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-9c3012-serving-cert\""
Sep 11 12:16:58.040584 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.040560    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"]
Sep 11 12:16:58.129977 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.129938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhmp7\" (UniqueName: \"kubernetes.io/projected/6b16436c-1294-484a-a828-2f0feccf68bf-kube-api-access-bhmp7\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.130168 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.130081    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-maas-api-tls\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.130168 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.130111    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b16436c-1294-484a-a828-2f0feccf68bf-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.130168 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.130149    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-metrics-certs\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.230941 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.230904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-maas-api-tls\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.230941 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.230941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b16436c-1294-484a-a828-2f0feccf68bf-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.231178 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.230971    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-metrics-certs\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.231178 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.231016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhmp7\" (UniqueName: \"kubernetes.io/projected/6b16436c-1294-484a-a828-2f0feccf68bf-kube-api-access-bhmp7\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.231178 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:16:58.231079    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-serving-cert: secret "maas-api-e2e-ipp-a-9c3012-serving-cert" not found
Sep 11 12:16:58.231178 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:16:58.231161    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-maas-api-tls podName:6b16436c-1294-484a-a828-2f0feccf68bf nodeName:}" failed. No retries permitted until 2026-09-11 12:16:58.731140666 +0000 UTC m=+1527.231860912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-maas-api-tls") pod "maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m" (UID: "6b16436c-1294-484a-a828-2f0feccf68bf") : secret "maas-api-e2e-ipp-a-9c3012-serving-cert" not found
Sep 11 12:16:58.231819 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.231794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b16436c-1294-484a-a828-2f0feccf68bf-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.233481 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.233461    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-metrics-certs\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.238881 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.238858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhmp7\" (UniqueName: \"kubernetes.io/projected/6b16436c-1294-484a-a828-2f0feccf68bf-kube-api-access-bhmp7\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.736185 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.736150    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-maas-api-tls\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.738699 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.738667    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-maas-api-tls\") pod \"maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:58.950027 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:58.949991    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:16:59.222871 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:59.222846    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"]
Sep 11 12:16:59.225092 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:16:59.225064    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b16436c_1294_484a_a828_2f0feccf68bf.slice/crio-0e36ede61f6e9de058d53887f9f2dacb7302bfa55ad5fe5d87e6312bca4a4241 WatchSource:0}: Error finding container 0e36ede61f6e9de058d53887f9f2dacb7302bfa55ad5fe5d87e6312bca4a4241: Status 404 returned error can't find the container with id 0e36ede61f6e9de058d53887f9f2dacb7302bfa55ad5fe5d87e6312bca4a4241
Sep 11 12:16:59.728496 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:16:59.728458    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m" event={"ID":"6b16436c-1294-484a-a828-2f0feccf68bf","Type":"ContainerStarted","Data":"0e36ede61f6e9de058d53887f9f2dacb7302bfa55ad5fe5d87e6312bca4a4241"}
Sep 11 12:17:00.735320 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:00.735269    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m" event={"ID":"6b16436c-1294-484a-a828-2f0feccf68bf","Type":"ContainerStarted","Data":"afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4"}
Sep 11 12:17:00.737716 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:00.735449    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:17:00.754453 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:00.754396    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m" podStartSLOduration=2.24798868 podStartE2EDuration="2.75437977s" totalImagesPullingTime="506.39109ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:17:00.752929632 +0000 UTC m=+1529.253649900" watchObservedRunningTime="2026-09-11 12:17:00.75437977 +0000 UTC m=+1529.255100037"
Sep 11 12:17:06.745799 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:06.745771    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:17:27.221164 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.221126    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"]
Sep 11 12:17:27.225184 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.225161    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.227191 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.227166    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-6d324e-serving-cert\""
Sep 11 12:17:27.234951 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.234896    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"]
Sep 11 12:17:27.313422 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.313386    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9eddbed-32d7-4049-be1d-44284a28f89c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.313606 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.313433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6b7b\" (UniqueName: \"kubernetes.io/projected/a9eddbed-32d7-4049-be1d-44284a28f89c-kube-api-access-m6b7b\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.313606 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.313527    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-metrics-certs\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.313606 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.313544    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.414828 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.414792    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6b7b\" (UniqueName: \"kubernetes.io/projected/a9eddbed-32d7-4049-be1d-44284a28f89c-kube-api-access-m6b7b\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.415007 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.414870    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-metrics-certs\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.415007 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.414891    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.415007 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.414941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9eddbed-32d7-4049-be1d-44284a28f89c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.415134 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:17:27.415047    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-serving-cert: secret "maas-api-e2e-ipp-b-6d324e-serving-cert" not found
Sep 11 12:17:27.415134 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:17:27.415130    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-maas-api-tls podName:a9eddbed-32d7-4049-be1d-44284a28f89c nodeName:}" failed. No retries permitted until 2026-09-11 12:17:27.915108146 +0000 UTC m=+1556.415828398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-maas-api-tls") pod "maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4" (UID: "a9eddbed-32d7-4049-be1d-44284a28f89c") : secret "maas-api-e2e-ipp-b-6d324e-serving-cert" not found
Sep 11 12:17:27.415653 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.415629    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9eddbed-32d7-4049-be1d-44284a28f89c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.417368 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.417348    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-metrics-certs\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.422657 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.422636    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6b7b\" (UniqueName: \"kubernetes.io/projected/a9eddbed-32d7-4049-be1d-44284a28f89c-kube-api-access-m6b7b\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.920182 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.920143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:27.922618 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:27.922589    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:28.138913 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:28.138876    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:28.419043 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:28.419019    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"]
Sep 11 12:17:28.424350 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:17:28.422158    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9eddbed_32d7_4049_be1d_44284a28f89c.slice/crio-11c7c886ea55a01f71d6a37074c9515a431cb3708dbcf6690d18c4004fa4f729 WatchSource:0}: Error finding container 11c7c886ea55a01f71d6a37074c9515a431cb3708dbcf6690d18c4004fa4f729: Status 404 returned error can't find the container with id 11c7c886ea55a01f71d6a37074c9515a431cb3708dbcf6690d18c4004fa4f729
Sep 11 12:17:28.878004 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:28.877969    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4" event={"ID":"a9eddbed-32d7-4049-be1d-44284a28f89c","Type":"ContainerStarted","Data":"11c7c886ea55a01f71d6a37074c9515a431cb3708dbcf6690d18c4004fa4f729"}
Sep 11 12:17:29.884258 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:29.884222    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4" event={"ID":"a9eddbed-32d7-4049-be1d-44284a28f89c","Type":"ContainerStarted","Data":"cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52"}
Sep 11 12:17:29.884695 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:29.884297    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:29.902478 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:29.902415    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4" podStartSLOduration=2.394343138 podStartE2EDuration="2.902399019s" totalImagesPullingTime="508.055881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:17:29.900694609 +0000 UTC m=+1558.401414867" watchObservedRunningTime="2026-09-11 12:17:29.902399019 +0000 UTC m=+1558.403119302"
Sep 11 12:17:35.895935 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:35.895906    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:17:55.885179 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:17:55.885151    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4mnp2_e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d/payload-processing/0.log"
Sep 11 12:18:26.522334 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:26.522219    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4mnp2_e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d/payload-processing/0.log"
Sep 11 12:18:31.639857 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.639819    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"]
Sep 11 12:18:31.643697 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.643672    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.645977 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.645951    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-49995f-serving-cert\""
Sep 11 12:18:31.654722 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.654657    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"]
Sep 11 12:18:31.744071 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.744032    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49m6l\" (UniqueName: \"kubernetes.io/projected/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-kube-api-access-49m6l\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.744245 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.744140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.744245 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.744162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.744245 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.744198    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.845470 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.845429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.845470 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.845470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.845675 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.845494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.845675 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.845572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49m6l\" (UniqueName: \"kubernetes.io/projected/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-kube-api-access-49m6l\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.845675 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:18:31.845656    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-serving-cert: secret "maas-api-e2e-ipp-cleanup-49995f-serving-cert" not found
Sep 11 12:18:31.845780 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:18:31.845728    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-maas-api-tls podName:ffb751bb-bdb8-4ecd-bf0f-559cae710f8b nodeName:}" failed. No retries permitted until 2026-09-11 12:18:32.345709839 +0000 UTC m=+1620.846430087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt" (UID: "ffb751bb-bdb8-4ecd-bf0f-559cae710f8b") : secret "maas-api-e2e-ipp-cleanup-49995f-serving-cert" not found
Sep 11 12:18:31.846187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.846165    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.847954 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.847933    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:31.856242 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:31.856218    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49m6l\" (UniqueName: \"kubernetes.io/projected/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-kube-api-access-49m6l\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:32.350975 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:32.350932    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:32.353593 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:32.353564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:32.557676 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:32.557634    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:32.833276 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:32.833248    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"]
Sep 11 12:18:32.835373 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:18:32.835345    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb751bb_bdb8_4ecd_bf0f_559cae710f8b.slice/crio-2956082cefd1eb6c6839840f4a3184bb3ce594fdcee93826c3a893f7cb0c5169 WatchSource:0}: Error finding container 2956082cefd1eb6c6839840f4a3184bb3ce594fdcee93826c3a893f7cb0c5169: Status 404 returned error can't find the container with id 2956082cefd1eb6c6839840f4a3184bb3ce594fdcee93826c3a893f7cb0c5169
Sep 11 12:18:33.207237 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:33.207208    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt" event={"ID":"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b","Type":"ContainerStarted","Data":"2956082cefd1eb6c6839840f4a3184bb3ce594fdcee93826c3a893f7cb0c5169"}
Sep 11 12:18:34.213297 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:34.213264    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt" event={"ID":"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b","Type":"ContainerStarted","Data":"7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280"}
Sep 11 12:18:34.213747 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:34.213353    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:18:34.231357 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:34.231289    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt" podStartSLOduration=2.868409459 podStartE2EDuration="3.231275141s" totalImagesPullingTime="362.865682ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:18:34.229875006 +0000 UTC m=+1622.730595286" watchObservedRunningTime="2026-09-11 12:18:34.231275141 +0000 UTC m=+1622.731995408"
Sep 11 12:18:40.225044 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:18:40.225013    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:19:02.085265 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.085231    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"]
Sep 11 12:19:02.085831 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.085484    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt" podUID="ffb751bb-bdb8-4ecd-bf0f-559cae710f8b" containerName="maas-api" containerID="cri-o://7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280" gracePeriod=30
Sep 11 12:19:02.343194 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.343135    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:19:02.356069 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.356041    2562 generic.go:356] "Generic (PLEG): container finished" podID="ffb751bb-bdb8-4ecd-bf0f-559cae710f8b" containerID="7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280" exitCode=0
Sep 11 12:19:02.356194 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.356090    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"
Sep 11 12:19:02.356194 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.356091    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt" event={"ID":"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b","Type":"ContainerDied","Data":"7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280"}
Sep 11 12:19:02.356276 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.356220    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt" event={"ID":"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b","Type":"ContainerDied","Data":"2956082cefd1eb6c6839840f4a3184bb3ce594fdcee93826c3a893f7cb0c5169"}
Sep 11 12:19:02.356276 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.356253    2562 scope.go:117] "RemoveContainer" containerID="7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280"
Sep 11 12:19:02.357007 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.356928    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49m6l\" (UniqueName: \"kubernetes.io/projected/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-kube-api-access-49m6l\") pod \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") "
Sep 11 12:19:02.358956 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.358933    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-kube-api-access-49m6l" (OuterVolumeSpecName: "kube-api-access-49m6l") pod "ffb751bb-bdb8-4ecd-bf0f-559cae710f8b" (UID: "ffb751bb-bdb8-4ecd-bf0f-559cae710f8b"). InnerVolumeSpecName "kube-api-access-49m6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:19:02.374995 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.374963    2562 scope.go:117] "RemoveContainer" containerID="7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280"
Sep 11 12:19:02.375412 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:19:02.375385    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280\": container with ID starting with 7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280 not found: ID does not exist" containerID="7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280"
Sep 11 12:19:02.375490 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.375424    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280"} err="failed to get container status \"7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280\": rpc error: code = NotFound desc = could not find container \"7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280\": container with ID starting with 7ebceaf76643364966493e740803f85f57c5772a452d6594be208b56ab99b280 not found: ID does not exist"
Sep 11 12:19:02.457523 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.457487    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-openshift-service-ca\") pod \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") "
Sep 11 12:19:02.457715 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.457552    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-maas-api-tls\") pod \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") "
Sep 11 12:19:02.457715 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.457578    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-metrics-certs\") pod \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\" (UID: \"ffb751bb-bdb8-4ecd-bf0f-559cae710f8b\") "
Sep 11 12:19:02.457935 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.457907    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ffb751bb-bdb8-4ecd-bf0f-559cae710f8b" (UID: "ffb751bb-bdb8-4ecd-bf0f-559cae710f8b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:19:02.458031 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.457989    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49m6l\" (UniqueName: \"kubernetes.io/projected/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-kube-api-access-49m6l\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:19:02.460096 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.460069    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ffb751bb-bdb8-4ecd-bf0f-559cae710f8b" (UID: "ffb751bb-bdb8-4ecd-bf0f-559cae710f8b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:19:02.460197 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.460098    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ffb751bb-bdb8-4ecd-bf0f-559cae710f8b" (UID: "ffb751bb-bdb8-4ecd-bf0f-559cae710f8b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:19:02.558692 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.558652    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:19:02.558692 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.558688    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:19:02.558692 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.558697    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:19:02.677838 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.677805    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"]
Sep 11 12:19:02.685643 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:02.685617    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-49995f-6dd56f9785-f5cwt"]
Sep 11 12:19:04.032092 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:04.032057    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffb751bb-bdb8-4ecd-bf0f-559cae710f8b" path="/var/lib/kubelet/pods/ffb751bb-bdb8-4ecd-bf0f-559cae710f8b/volumes"
Sep 11 12:19:27.885453 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:27.885414    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"]
Sep 11 12:19:27.885930 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:27.885630    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m" podUID="6b16436c-1294-484a-a828-2f0feccf68bf" containerName="maas-api" containerID="cri-o://afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4" gracePeriod=30
Sep 11 12:19:28.148602 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.148533    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:19:28.297371 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.297295    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-metrics-certs\") pod \"6b16436c-1294-484a-a828-2f0feccf68bf\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") "
Sep 11 12:19:28.297586 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.297466    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-maas-api-tls\") pod \"6b16436c-1294-484a-a828-2f0feccf68bf\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") "
Sep 11 12:19:28.297653 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.297614    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b16436c-1294-484a-a828-2f0feccf68bf-openshift-service-ca\") pod \"6b16436c-1294-484a-a828-2f0feccf68bf\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") "
Sep 11 12:19:28.297761 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.297739    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhmp7\" (UniqueName: \"kubernetes.io/projected/6b16436c-1294-484a-a828-2f0feccf68bf-kube-api-access-bhmp7\") pod \"6b16436c-1294-484a-a828-2f0feccf68bf\" (UID: \"6b16436c-1294-484a-a828-2f0feccf68bf\") "
Sep 11 12:19:28.298862 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.298807    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b16436c-1294-484a-a828-2f0feccf68bf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b16436c-1294-484a-a828-2f0feccf68bf" (UID: "6b16436c-1294-484a-a828-2f0feccf68bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:19:28.299982 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.299950    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b16436c-1294-484a-a828-2f0feccf68bf" (UID: "6b16436c-1294-484a-a828-2f0feccf68bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:19:28.300123 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.300047    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b16436c-1294-484a-a828-2f0feccf68bf-kube-api-access-bhmp7" (OuterVolumeSpecName: "kube-api-access-bhmp7") pod "6b16436c-1294-484a-a828-2f0feccf68bf" (UID: "6b16436c-1294-484a-a828-2f0feccf68bf"). InnerVolumeSpecName "kube-api-access-bhmp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:19:28.300187 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.300159    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6b16436c-1294-484a-a828-2f0feccf68bf" (UID: "6b16436c-1294-484a-a828-2f0feccf68bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:19:28.399373 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.399283    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b16436c-1294-484a-a828-2f0feccf68bf-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:19:28.399373 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.399322    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhmp7\" (UniqueName: \"kubernetes.io/projected/6b16436c-1294-484a-a828-2f0feccf68bf-kube-api-access-bhmp7\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:19:28.399373 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.399334    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:19:28.399373 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.399343    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b16436c-1294-484a-a828-2f0feccf68bf-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:19:28.493260 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.493228    2562 generic.go:356] "Generic (PLEG): container finished" podID="6b16436c-1294-484a-a828-2f0feccf68bf" containerID="afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4" exitCode=0
Sep 11 12:19:28.493449 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.493270    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"
Sep 11 12:19:28.493449 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.493330    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m" event={"ID":"6b16436c-1294-484a-a828-2f0feccf68bf","Type":"ContainerDied","Data":"afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4"}
Sep 11 12:19:28.493449 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.493384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m" event={"ID":"6b16436c-1294-484a-a828-2f0feccf68bf","Type":"ContainerDied","Data":"0e36ede61f6e9de058d53887f9f2dacb7302bfa55ad5fe5d87e6312bca4a4241"}
Sep 11 12:19:28.493449 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.493408    2562 scope.go:117] "RemoveContainer" containerID="afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4"
Sep 11 12:19:28.503901 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.503883    2562 scope.go:117] "RemoveContainer" containerID="afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4"
Sep 11 12:19:28.504198 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:19:28.504180    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4\": container with ID starting with afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4 not found: ID does not exist" containerID="afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4"
Sep 11 12:19:28.504286 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.504206    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4"} err="failed to get container status \"afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4\": rpc error: code = NotFound desc = could not find container \"afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4\": container with ID starting with afac50e357110fe28da9a7bf369e697e0ab8c784e85733e1a6b35926215738a4 not found: ID does not exist"
Sep 11 12:19:28.514525 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.514499    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"]
Sep 11 12:19:28.520968 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:28.520945    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-9c3012-c75dcd68d-mv48m"]
Sep 11 12:19:30.025296 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:19:30.025261    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b16436c-1294-484a-a828-2f0feccf68bf" path="/var/lib/kubelet/pods/6b16436c-1294-484a-a828-2f0feccf68bf/volumes"
Sep 11 12:20:32.746828 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:32.746789    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"]
Sep 11 12:20:32.747431 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:32.747085    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4" podUID="a9eddbed-32d7-4049-be1d-44284a28f89c" containerName="maas-api" containerID="cri-o://cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52" gracePeriod=30
Sep 11 12:20:32.998659 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:32.998602    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:20:33.098525 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.098494    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-maas-api-tls\") pod \"a9eddbed-32d7-4049-be1d-44284a28f89c\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") "
Sep 11 12:20:33.098702 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.098533    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-metrics-certs\") pod \"a9eddbed-32d7-4049-be1d-44284a28f89c\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") "
Sep 11 12:20:33.098702 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.098574    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9eddbed-32d7-4049-be1d-44284a28f89c-openshift-service-ca\") pod \"a9eddbed-32d7-4049-be1d-44284a28f89c\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") "
Sep 11 12:20:33.098702 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.098658    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6b7b\" (UniqueName: \"kubernetes.io/projected/a9eddbed-32d7-4049-be1d-44284a28f89c-kube-api-access-m6b7b\") pod \"a9eddbed-32d7-4049-be1d-44284a28f89c\" (UID: \"a9eddbed-32d7-4049-be1d-44284a28f89c\") "
Sep 11 12:20:33.099110 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.099079    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9eddbed-32d7-4049-be1d-44284a28f89c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9eddbed-32d7-4049-be1d-44284a28f89c" (UID: "a9eddbed-32d7-4049-be1d-44284a28f89c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:20:33.100886 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.100853    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a9eddbed-32d7-4049-be1d-44284a28f89c" (UID: "a9eddbed-32d7-4049-be1d-44284a28f89c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:20:33.100985 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.100928    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9eddbed-32d7-4049-be1d-44284a28f89c-kube-api-access-m6b7b" (OuterVolumeSpecName: "kube-api-access-m6b7b") pod "a9eddbed-32d7-4049-be1d-44284a28f89c" (UID: "a9eddbed-32d7-4049-be1d-44284a28f89c"). InnerVolumeSpecName "kube-api-access-m6b7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:20:33.101049 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.101035    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9eddbed-32d7-4049-be1d-44284a28f89c" (UID: "a9eddbed-32d7-4049-be1d-44284a28f89c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:20:33.199751 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.199715    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:20:33.199751 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.199744    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9eddbed-32d7-4049-be1d-44284a28f89c-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:20:33.199751 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.199756    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9eddbed-32d7-4049-be1d-44284a28f89c-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:20:33.199998 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.199766    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6b7b\" (UniqueName: \"kubernetes.io/projected/a9eddbed-32d7-4049-be1d-44284a28f89c-kube-api-access-m6b7b\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:20:33.817676 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.817638    2562 generic.go:356] "Generic (PLEG): container finished" podID="a9eddbed-32d7-4049-be1d-44284a28f89c" containerID="cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52" exitCode=0
Sep 11 12:20:33.818175 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.817691    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"
Sep 11 12:20:33.818175 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.817727    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4" event={"ID":"a9eddbed-32d7-4049-be1d-44284a28f89c","Type":"ContainerDied","Data":"cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52"}
Sep 11 12:20:33.818175 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.817795    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4" event={"ID":"a9eddbed-32d7-4049-be1d-44284a28f89c","Type":"ContainerDied","Data":"11c7c886ea55a01f71d6a37074c9515a431cb3708dbcf6690d18c4004fa4f729"}
Sep 11 12:20:33.818175 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.817830    2562 scope.go:117] "RemoveContainer" containerID="cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52"
Sep 11 12:20:33.828125 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.828106    2562 scope.go:117] "RemoveContainer" containerID="cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52"
Sep 11 12:20:33.828448 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:20:33.828426    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52\": container with ID starting with cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52 not found: ID does not exist" containerID="cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52"
Sep 11 12:20:33.828532 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.828458    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52"} err="failed to get container status \"cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52\": rpc error: code = NotFound desc = could not find container \"cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52\": container with ID starting with cd232ad83d7f56cec6d2a51117cacd90e2af721201cf767f25ef787e30e7ac52 not found: ID does not exist"
Sep 11 12:20:33.837810 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.837783    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"]
Sep 11 12:20:33.841678 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:33.841654    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6d324e-76bdb56fc7-p59m4"]
Sep 11 12:20:34.025116 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:34.025079    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9eddbed-32d7-4049-be1d-44284a28f89c" path="/var/lib/kubelet/pods/a9eddbed-32d7-4049-be1d-44284a28f89c/volumes"
Sep 11 12:20:50.354233 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.354198    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"]
Sep 11 12:20:50.354734 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.354485    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j" podUID="79ca0d64-6f85-4f7a-ab68-65f6f313cc1d" containerName="maas-api" containerID="cri-o://6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699" gracePeriod=30
Sep 11 12:20:50.408465 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.408427    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"]
Sep 11 12:20:50.408721 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.408690    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8" podUID="441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2" containerName="payload-processing" containerID="cri-o://fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1" gracePeriod=30
Sep 11 12:20:50.669183 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.669154    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:20:50.701622 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.701590    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:20:50.766622 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.766586    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-openshift-service-ca\") pod \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") "
Sep 11 12:20:50.766622 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.766625    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-plugins-config-volume\") pod \"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2\" (UID: \"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2\") "
Sep 11 12:20:50.766833 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.766661    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-metrics-certs\") pod \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") "
Sep 11 12:20:50.766833 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.766693    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtcd8\" (UniqueName: \"kubernetes.io/projected/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-kube-api-access-gtcd8\") pod \"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2\" (UID: \"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2\") "
Sep 11 12:20:50.766833 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.766764    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-maas-api-tls\") pod \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") "
Sep 11 12:20:50.766833 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.766802    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xtnv\" (UniqueName: \"kubernetes.io/projected/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-kube-api-access-9xtnv\") pod \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\" (UID: \"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d\") "
Sep 11 12:20:50.767082 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.767050    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79ca0d64-6f85-4f7a-ab68-65f6f313cc1d" (UID: "79ca0d64-6f85-4f7a-ab68-65f6f313cc1d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:20:50.767164 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.767100    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2" (UID: "441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:20:50.769021 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.768985    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-kube-api-access-9xtnv" (OuterVolumeSpecName: "kube-api-access-9xtnv") pod "79ca0d64-6f85-4f7a-ab68-65f6f313cc1d" (UID: "79ca0d64-6f85-4f7a-ab68-65f6f313cc1d"). InnerVolumeSpecName "kube-api-access-9xtnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:20:50.769124 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.769042    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "79ca0d64-6f85-4f7a-ab68-65f6f313cc1d" (UID: "79ca0d64-6f85-4f7a-ab68-65f6f313cc1d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:20:50.769124 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.769052    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-kube-api-access-gtcd8" (OuterVolumeSpecName: "kube-api-access-gtcd8") pod "441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2" (UID: "441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2"). InnerVolumeSpecName "kube-api-access-gtcd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:20:50.769124 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.769061    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79ca0d64-6f85-4f7a-ab68-65f6f313cc1d" (UID: "79ca0d64-6f85-4f7a-ab68-65f6f313cc1d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:20:50.867805 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.867711    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xtnv\" (UniqueName: \"kubernetes.io/projected/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-kube-api-access-9xtnv\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:20:50.867805 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.867744    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:20:50.867805 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.867755    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:20:50.867805 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.867766    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:20:50.867805 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.867774    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtcd8\" (UniqueName: \"kubernetes.io/projected/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2-kube-api-access-gtcd8\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:20:50.867805 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.867783    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:20:50.907083 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.907046    2562 generic.go:356] "Generic (PLEG): container finished" podID="79ca0d64-6f85-4f7a-ab68-65f6f313cc1d" containerID="6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699" exitCode=0
Sep 11 12:20:50.907257 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.907100    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"
Sep 11 12:20:50.907257 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.907130    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j" event={"ID":"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d","Type":"ContainerDied","Data":"6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699"}
Sep 11 12:20:50.907257 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.907204    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j" event={"ID":"79ca0d64-6f85-4f7a-ab68-65f6f313cc1d","Type":"ContainerDied","Data":"d01e811127d513cea440e0c7312b2d48b55baf6d439601b89f32e8502c6d31ae"}
Sep 11 12:20:50.907257 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.907242    2562 scope.go:117] "RemoveContainer" containerID="6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699"
Sep 11 12:20:50.908525 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.908499    2562 generic.go:356] "Generic (PLEG): container finished" podID="441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2" containerID="fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1" exitCode=0
Sep 11 12:20:50.908671 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.908529    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8" event={"ID":"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2","Type":"ContainerDied","Data":"fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1"}
Sep 11 12:20:50.908671 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.908580    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8" event={"ID":"441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2","Type":"ContainerDied","Data":"285ad4b5698939a2e2188f5087d73f6fbc324873dc83a706c8c9360ef4681ac8"}
Sep 11 12:20:50.908671 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.908546    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"
Sep 11 12:20:50.918326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.918283    2562 scope.go:117] "RemoveContainer" containerID="6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699"
Sep 11 12:20:50.918694 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:20:50.918664    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699\": container with ID starting with 6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699 not found: ID does not exist" containerID="6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699"
Sep 11 12:20:50.918778 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.918708    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699"} err="failed to get container status \"6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699\": rpc error: code = NotFound desc = could not find container \"6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699\": container with ID starting with 6ad91786191988d3ed29407a3af2472ac97f28015e010c3ec8baa249758c8699 not found: ID does not exist"
Sep 11 12:20:50.918778 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.918734    2562 scope.go:117] "RemoveContainer" containerID="fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1"
Sep 11 12:20:50.927521 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.927503    2562 scope.go:117] "RemoveContainer" containerID="fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1"
Sep 11 12:20:50.927813 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:20:50.927796    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1\": container with ID starting with fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1 not found: ID does not exist" containerID="fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1"
Sep 11 12:20:50.927862 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.927821    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1"} err="failed to get container status \"fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1\": rpc error: code = NotFound desc = could not find container \"fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1\": container with ID starting with fc265575741e5e93334fea75451b817f27f7316611f29ec2e96804f3fcaf07f1 not found: ID does not exist"
Sep 11 12:20:50.934408 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.934385    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"]
Sep 11 12:20:50.940652 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.940631    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-88968d-68dc9646d8-ghf6j"]
Sep 11 12:20:50.950492 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.950466    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"]
Sep 11 12:20:50.954605 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:50.954575    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-88968d-5f9456cfb5-cmlt8"]
Sep 11 12:20:52.032859 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:52.032818    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2" path="/var/lib/kubelet/pods/441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2/volumes"
Sep 11 12:20:52.033410 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:20:52.033391    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79ca0d64-6f85-4f7a-ab68-65f6f313cc1d" path="/var/lib/kubelet/pods/79ca0d64-6f85-4f7a-ab68-65f6f313cc1d/volumes"
Sep 11 12:21:08.245427 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.245389    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"]
Sep 11 12:21:08.246003 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.245643    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx" podUID="44d41d00-f9b6-46b7-bff9-b8cecf425b75" containerName="maas-api" containerID="cri-o://bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4" gracePeriod=30
Sep 11 12:21:08.301335 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.301285    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"]
Sep 11 12:21:08.301541 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.301513    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5" podUID="2a108bfc-713f-4d3c-8c02-64ed9cf0031e" containerName="payload-processing" containerID="cri-o://6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f" gracePeriod=30
Sep 11 12:21:08.547692 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.547667    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:21:08.578680 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.578655    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:21:08.741297 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.741256    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n48zl\" (UniqueName: \"kubernetes.io/projected/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-kube-api-access-n48zl\") pod \"2a108bfc-713f-4d3c-8c02-64ed9cf0031e\" (UID: \"2a108bfc-713f-4d3c-8c02-64ed9cf0031e\") "
Sep 11 12:21:08.741297 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.741299    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwn7b\" (UniqueName: \"kubernetes.io/projected/44d41d00-f9b6-46b7-bff9-b8cecf425b75-kube-api-access-dwn7b\") pod \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") "
Sep 11 12:21:08.741582 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.741337    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-metrics-certs\") pod \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") "
Sep 11 12:21:08.741582 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.741373    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-plugins-config-volume\") pod \"2a108bfc-713f-4d3c-8c02-64ed9cf0031e\" (UID: \"2a108bfc-713f-4d3c-8c02-64ed9cf0031e\") "
Sep 11 12:21:08.741582 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.741452    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d41d00-f9b6-46b7-bff9-b8cecf425b75-openshift-service-ca\") pod \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") "
Sep 11 12:21:08.741582 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.741481    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-maas-api-tls\") pod \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\" (UID: \"44d41d00-f9b6-46b7-bff9-b8cecf425b75\") "
Sep 11 12:21:08.741880 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.741840    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d41d00-f9b6-46b7-bff9-b8cecf425b75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44d41d00-f9b6-46b7-bff9-b8cecf425b75" (UID: "44d41d00-f9b6-46b7-bff9-b8cecf425b75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:21:08.741880 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.741854    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a108bfc-713f-4d3c-8c02-64ed9cf0031e" (UID: "2a108bfc-713f-4d3c-8c02-64ed9cf0031e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:21:08.743598 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.743574    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-kube-api-access-n48zl" (OuterVolumeSpecName: "kube-api-access-n48zl") pod "2a108bfc-713f-4d3c-8c02-64ed9cf0031e" (UID: "2a108bfc-713f-4d3c-8c02-64ed9cf0031e"). InnerVolumeSpecName "kube-api-access-n48zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:21:08.743598 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.743585    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "44d41d00-f9b6-46b7-bff9-b8cecf425b75" (UID: "44d41d00-f9b6-46b7-bff9-b8cecf425b75"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:21:08.743871 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.743848    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44d41d00-f9b6-46b7-bff9-b8cecf425b75-kube-api-access-dwn7b" (OuterVolumeSpecName: "kube-api-access-dwn7b") pod "44d41d00-f9b6-46b7-bff9-b8cecf425b75" (UID: "44d41d00-f9b6-46b7-bff9-b8cecf425b75"). InnerVolumeSpecName "kube-api-access-dwn7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:21:08.743948 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.743912    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44d41d00-f9b6-46b7-bff9-b8cecf425b75" (UID: "44d41d00-f9b6-46b7-bff9-b8cecf425b75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:21:08.843060 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.842958    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d41d00-f9b6-46b7-bff9-b8cecf425b75-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:21:08.843060 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.842992    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:21:08.843060 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.843006    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n48zl\" (UniqueName: \"kubernetes.io/projected/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-kube-api-access-n48zl\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:21:08.843060 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.843022    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwn7b\" (UniqueName: \"kubernetes.io/projected/44d41d00-f9b6-46b7-bff9-b8cecf425b75-kube-api-access-dwn7b\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:21:08.843060 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.843033    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44d41d00-f9b6-46b7-bff9-b8cecf425b75-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:21:08.843060 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:08.843046    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a108bfc-713f-4d3c-8c02-64ed9cf0031e-plugins-config-volume\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:21:09.003444 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.003406    2562 generic.go:356] "Generic (PLEG): container finished" podID="2a108bfc-713f-4d3c-8c02-64ed9cf0031e" containerID="6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f" exitCode=0
Sep 11 12:21:09.003639 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.003455    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"
Sep 11 12:21:09.003639 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.003526    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5" event={"ID":"2a108bfc-713f-4d3c-8c02-64ed9cf0031e","Type":"ContainerDied","Data":"6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f"}
Sep 11 12:21:09.003639 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.003576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5" event={"ID":"2a108bfc-713f-4d3c-8c02-64ed9cf0031e","Type":"ContainerDied","Data":"b2c49b115601fb01d3d49ef7eb9820a1ac06276afed5c40ec425ecc1dbe9ea0b"}
Sep 11 12:21:09.003639 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.003599    2562 scope.go:117] "RemoveContainer" containerID="6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f"
Sep 11 12:21:09.004949 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.004906    2562 generic.go:356] "Generic (PLEG): container finished" podID="44d41d00-f9b6-46b7-bff9-b8cecf425b75" containerID="bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4" exitCode=0
Sep 11 12:21:09.005054 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.004942    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx" event={"ID":"44d41d00-f9b6-46b7-bff9-b8cecf425b75","Type":"ContainerDied","Data":"bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4"}
Sep 11 12:21:09.005054 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.005008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx" event={"ID":"44d41d00-f9b6-46b7-bff9-b8cecf425b75","Type":"ContainerDied","Data":"c9e147511c6fff3313ccdc36e0f32079c78472b7c37d3970dc290833395eafce"}
Sep 11 12:21:09.005054 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.004952    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"
Sep 11 12:21:09.013499 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.013474    2562 scope.go:117] "RemoveContainer" containerID="6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f"
Sep 11 12:21:09.013814 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:21:09.013790    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f\": container with ID starting with 6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f not found: ID does not exist" containerID="6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f"
Sep 11 12:21:09.013912 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.013827    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f"} err="failed to get container status \"6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f\": rpc error: code = NotFound desc = could not find container \"6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f\": container with ID starting with 6db1b9bf2227bc3a8ffe9e83e5fb045eddfb379849123df0ce3969974addc45f not found: ID does not exist"
Sep 11 12:21:09.013912 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.013851    2562 scope.go:117] "RemoveContainer" containerID="bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4"
Sep 11 12:21:09.022462 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.022435    2562 scope.go:117] "RemoveContainer" containerID="bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4"
Sep 11 12:21:09.022728 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:21:09.022710    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4\": container with ID starting with bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4 not found: ID does not exist" containerID="bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4"
Sep 11 12:21:09.022783 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.022735    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4"} err="failed to get container status \"bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4\": rpc error: code = NotFound desc = could not find container \"bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4\": container with ID starting with bd3f03a13df556326d6b84e614554241f30ac0086103a867941f8063bfd87cf4 not found: ID does not exist"
Sep 11 12:21:09.028706 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.028683    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"]
Sep 11 12:21:09.032887 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.032866    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx"]
Sep 11 12:21:09.041818 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.041789    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"]
Sep 11 12:21:09.045862 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:09.045840    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4d3f83-8686798969-wfgz5"]
Sep 11 12:21:10.024153 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:10.024115    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a108bfc-713f-4d3c-8c02-64ed9cf0031e" path="/var/lib/kubelet/pods/2a108bfc-713f-4d3c-8c02-64ed9cf0031e/volumes"
Sep 11 12:21:10.024551 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:10.024533    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44d41d00-f9b6-46b7-bff9-b8cecf425b75" path="/var/lib/kubelet/pods/44d41d00-f9b6-46b7-bff9-b8cecf425b75/volumes"
Sep 11 12:21:11.513978 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:11.513918    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4d3f83-6f6f576458-vzcdx" podUID="44d41d00-f9b6-46b7-bff9-b8cecf425b75" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.61:8443/health\": context deadline exceeded"
Sep 11 12:21:18.447010 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.446974    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-r5x4s"]
Sep 11 12:21:18.447509 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.447174    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7db788bf7f-r5x4s" podUID="3c3385b3-c3fa-43b8-9b66-8db58f532a6d" containerName="manager" containerID="cri-o://022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80" gracePeriod=30
Sep 11 12:21:18.681913 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.681885    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 12:21:18.736155 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.736072    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-webhook-cert\") pod \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") "
Sep 11 12:21:18.736155 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.736132    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-metrics-certs\") pod \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") "
Sep 11 12:21:18.736437 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.736261    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcb5z\" (UniqueName: \"kubernetes.io/projected/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-kube-api-access-mcb5z\") pod \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\" (UID: \"3c3385b3-c3fa-43b8-9b66-8db58f532a6d\") "
Sep 11 12:21:18.738367 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.738329    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-kube-api-access-mcb5z" (OuterVolumeSpecName: "kube-api-access-mcb5z") pod "3c3385b3-c3fa-43b8-9b66-8db58f532a6d" (UID: "3c3385b3-c3fa-43b8-9b66-8db58f532a6d"). InnerVolumeSpecName "kube-api-access-mcb5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:21:18.738662 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.738628    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3c3385b3-c3fa-43b8-9b66-8db58f532a6d" (UID: "3c3385b3-c3fa-43b8-9b66-8db58f532a6d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:21:18.738782 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.738766    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3c3385b3-c3fa-43b8-9b66-8db58f532a6d" (UID: "3c3385b3-c3fa-43b8-9b66-8db58f532a6d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:21:18.837883 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.837848    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-webhook-cert\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:21:18.837883 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.837877    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:21:18.837883 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:18.837887    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcb5z\" (UniqueName: \"kubernetes.io/projected/3c3385b3-c3fa-43b8-9b66-8db58f532a6d-kube-api-access-mcb5z\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:21:19.056389 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:19.056289    2562 generic.go:356] "Generic (PLEG): container finished" podID="3c3385b3-c3fa-43b8-9b66-8db58f532a6d" containerID="022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80" exitCode=0
Sep 11 12:21:19.056389 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:19.056347    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7db788bf7f-r5x4s"
Sep 11 12:21:19.056389 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:19.056355    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7db788bf7f-r5x4s" event={"ID":"3c3385b3-c3fa-43b8-9b66-8db58f532a6d","Type":"ContainerDied","Data":"022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80"}
Sep 11 12:21:19.056608 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:19.056395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7db788bf7f-r5x4s" event={"ID":"3c3385b3-c3fa-43b8-9b66-8db58f532a6d","Type":"ContainerDied","Data":"c897ccaeffadd9c60e28c3441a29c8b255d3c154b9785945e9e5199b340da770"}
Sep 11 12:21:19.056608 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:19.056417    2562 scope.go:117] "RemoveContainer" containerID="022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80"
Sep 11 12:21:19.065823 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:19.065803    2562 scope.go:117] "RemoveContainer" containerID="022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80"
Sep 11 12:21:19.066089 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:21:19.066069    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80\": container with ID starting with 022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80 not found: ID does not exist" containerID="022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80"
Sep 11 12:21:19.066129 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:19.066099    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80"} err="failed to get container status \"022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80\": rpc error: code = NotFound desc = could not find container \"022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80\": container with ID starting with 022d6e7f6fab8f660060343532d52fb58cca18c45fa313461b1e9463719e6b80 not found: ID does not exist"
Sep 11 12:21:19.077319 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:19.077282    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-r5x4s"]
Sep 11 12:21:19.078941 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:19.078918    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-r5x4s"]
Sep 11 12:21:20.025202 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:21:20.025154    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c3385b3-c3fa-43b8-9b66-8db58f532a6d" path="/var/lib/kubelet/pods/3c3385b3-c3fa-43b8-9b66-8db58f532a6d/volumes"
Sep 11 12:22:19.903004 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.902969    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7db788bf7f-lj2dz"]
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903355    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b16436c-1294-484a-a828-2f0feccf68bf" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903367    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b16436c-1294-484a-a828-2f0feccf68bf" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903387    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ca0d64-6f85-4f7a-ab68-65f6f313cc1d" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903393    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ca0d64-6f85-4f7a-ab68-65f6f313cc1d" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903400    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c3385b3-c3fa-43b8-9b66-8db58f532a6d" containerName="manager"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903406    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c3385b3-c3fa-43b8-9b66-8db58f532a6d" containerName="manager"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903412    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d41d00-f9b6-46b7-bff9-b8cecf425b75" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903416    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d41d00-f9b6-46b7-bff9-b8cecf425b75" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903424    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a108bfc-713f-4d3c-8c02-64ed9cf0031e" containerName="payload-processing"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903429    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a108bfc-713f-4d3c-8c02-64ed9cf0031e" containerName="payload-processing"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903435    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb751bb-bdb8-4ecd-bf0f-559cae710f8b" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903440    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb751bb-bdb8-4ecd-bf0f-559cae710f8b" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903449    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9eddbed-32d7-4049-be1d-44284a28f89c" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903454    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9eddbed-32d7-4049-be1d-44284a28f89c" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903464    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2" containerName="payload-processing"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903469    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2" containerName="payload-processing"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903527    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="441f8ca9-ac63-4db4-8f2b-cbb71a1a1bb2" containerName="payload-processing"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903535    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9eddbed-32d7-4049-be1d-44284a28f89c" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903543    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d41d00-f9b6-46b7-bff9-b8cecf425b75" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903554    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a108bfc-713f-4d3c-8c02-64ed9cf0031e" containerName="payload-processing"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903564    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb751bb-bdb8-4ecd-bf0f-559cae710f8b" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903573    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c3385b3-c3fa-43b8-9b66-8db58f532a6d" containerName="manager"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903581    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ca0d64-6f85-4f7a-ab68-65f6f313cc1d" containerName="maas-api"
Sep 11 12:22:19.905491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.903589    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b16436c-1294-484a-a828-2f0feccf68bf" containerName="maas-api"
Sep 11 12:22:19.906554 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.906537    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:19.909132 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.909105    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 12:22:19.909234 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.909131    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fcqsp\""
Sep 11 12:22:19.909234 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.909121    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 12:22:19.909234 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.909189    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 12:22:19.916740 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:19.916717    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-lj2dz"]
Sep 11 12:22:20.011948 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.011907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-webhook-cert\") pod \"maas-controller-7db788bf7f-lj2dz\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") " pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:20.011948 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.011949    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-metrics-certs\") pod \"maas-controller-7db788bf7f-lj2dz\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") " pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:20.012176 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.012007    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf7kl\" (UniqueName: \"kubernetes.io/projected/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-kube-api-access-jf7kl\") pod \"maas-controller-7db788bf7f-lj2dz\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") " pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:20.112756 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.112717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-webhook-cert\") pod \"maas-controller-7db788bf7f-lj2dz\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") " pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:20.112756 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.112757    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-metrics-certs\") pod \"maas-controller-7db788bf7f-lj2dz\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") " pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:20.112969 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.112801    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf7kl\" (UniqueName: \"kubernetes.io/projected/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-kube-api-access-jf7kl\") pod \"maas-controller-7db788bf7f-lj2dz\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") " pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:20.115251 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.115221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-webhook-cert\") pod \"maas-controller-7db788bf7f-lj2dz\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") " pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:20.115389 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.115253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-metrics-certs\") pod \"maas-controller-7db788bf7f-lj2dz\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") " pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:20.120505 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.120485    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf7kl\" (UniqueName: \"kubernetes.io/projected/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-kube-api-access-jf7kl\") pod \"maas-controller-7db788bf7f-lj2dz\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") " pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:20.217279 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.217240    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:20.489981 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.489724    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-lj2dz"]
Sep 11 12:22:20.492891 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:22:20.492864    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e2ef05_010b_407d_b74f_156dc8a8bfa9.slice/crio-1af6b0e54eaef7b60412cde5024b85fc4d08d79eb40250570249740a636e0ad6 WatchSource:0}: Error finding container 1af6b0e54eaef7b60412cde5024b85fc4d08d79eb40250570249740a636e0ad6: Status 404 returned error can't find the container with id 1af6b0e54eaef7b60412cde5024b85fc4d08d79eb40250570249740a636e0ad6
Sep 11 12:22:20.494192 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:20.494174    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:22:21.349096 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:21.349008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7db788bf7f-lj2dz" event={"ID":"e7e2ef05-010b-407d-b74f-156dc8a8bfa9","Type":"ContainerStarted","Data":"73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b"}
Sep 11 12:22:21.349096 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:21.349058    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7db788bf7f-lj2dz" event={"ID":"e7e2ef05-010b-407d-b74f-156dc8a8bfa9","Type":"ContainerStarted","Data":"1af6b0e54eaef7b60412cde5024b85fc4d08d79eb40250570249740a636e0ad6"}
Sep 11 12:22:21.349535 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:21.349144    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:21.367140 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:21.367085    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7db788bf7f-lj2dz" podStartSLOduration=1.8319070480000001 podStartE2EDuration="2.367067492s" totalImagesPullingTime="535.160444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:22:21.364705409 +0000 UTC m=+1849.865425678" watchObservedRunningTime="2026-09-11 12:22:21.367067492 +0000 UTC m=+1849.867787759"
Sep 11 12:22:32.356341 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:32.356289    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:41.790411 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:41.790372    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-lj2dz"]
Sep 11 12:22:41.790917 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:41.790633    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7db788bf7f-lj2dz" podUID="e7e2ef05-010b-407d-b74f-156dc8a8bfa9" containerName="manager" containerID="cri-o://73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b" gracePeriod=30
Sep 11 12:22:42.036218 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.036192    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:42.101561 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.101479    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-webhook-cert\") pod \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") "
Sep 11 12:22:42.101561 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.101541    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-metrics-certs\") pod \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") "
Sep 11 12:22:42.101737 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.101580    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf7kl\" (UniqueName: \"kubernetes.io/projected/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-kube-api-access-jf7kl\") pod \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\" (UID: \"e7e2ef05-010b-407d-b74f-156dc8a8bfa9\") "
Sep 11 12:22:42.103762 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.103728    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e7e2ef05-010b-407d-b74f-156dc8a8bfa9" (UID: "e7e2ef05-010b-407d-b74f-156dc8a8bfa9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:22:42.103903 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.103862    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-kube-api-access-jf7kl" (OuterVolumeSpecName: "kube-api-access-jf7kl") pod "e7e2ef05-010b-407d-b74f-156dc8a8bfa9" (UID: "e7e2ef05-010b-407d-b74f-156dc8a8bfa9"). InnerVolumeSpecName "kube-api-access-jf7kl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:22:42.103966 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.103944    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e7e2ef05-010b-407d-b74f-156dc8a8bfa9" (UID: "e7e2ef05-010b-407d-b74f-156dc8a8bfa9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:22:42.202661 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.202622    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-webhook-cert\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:22:42.202661 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.202653    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:22:42.202661 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.202662    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf7kl\" (UniqueName: \"kubernetes.io/projected/e7e2ef05-010b-407d-b74f-156dc8a8bfa9-kube-api-access-jf7kl\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:22:42.446266 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.446230    2562 generic.go:356] "Generic (PLEG): container finished" podID="e7e2ef05-010b-407d-b74f-156dc8a8bfa9" containerID="73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b" exitCode=0
Sep 11 12:22:42.446463 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.446276    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7db788bf7f-lj2dz"
Sep 11 12:22:42.446463 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.446336    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7db788bf7f-lj2dz" event={"ID":"e7e2ef05-010b-407d-b74f-156dc8a8bfa9","Type":"ContainerDied","Data":"73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b"}
Sep 11 12:22:42.446463 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.446394    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7db788bf7f-lj2dz" event={"ID":"e7e2ef05-010b-407d-b74f-156dc8a8bfa9","Type":"ContainerDied","Data":"1af6b0e54eaef7b60412cde5024b85fc4d08d79eb40250570249740a636e0ad6"}
Sep 11 12:22:42.446463 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.446424    2562 scope.go:117] "RemoveContainer" containerID="73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b"
Sep 11 12:22:42.455379 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.455360    2562 scope.go:117] "RemoveContainer" containerID="73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b"
Sep 11 12:22:42.455667 ip-10-0-136-222 kubenswrapper[2562]: E0911 12:22:42.455650    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b\": container with ID starting with 73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b not found: ID does not exist" containerID="73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b"
Sep 11 12:22:42.455718 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.455675    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b"} err="failed to get container status \"73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b\": rpc error: code = NotFound desc = could not find container \"73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b\": container with ID starting with 73ccbca6439167abaa7be38b664033a3723e399fee08f8223edaffb1fcbc581b not found: ID does not exist"
Sep 11 12:22:42.467386 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.467361    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-lj2dz"]
Sep 11 12:22:42.473721 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:42.473698    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-lj2dz"]
Sep 11 12:22:43.308252 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.308216    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7db788bf7f-8n2pn"]
Sep 11 12:22:43.308820 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.308802    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e2ef05-010b-407d-b74f-156dc8a8bfa9" containerName="manager"
Sep 11 12:22:43.308873 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.308822    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e2ef05-010b-407d-b74f-156dc8a8bfa9" containerName="manager"
Sep 11 12:22:43.308939 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.308929    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7e2ef05-010b-407d-b74f-156dc8a8bfa9" containerName="manager"
Sep 11 12:22:43.313329 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.313282    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.315756 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.315727    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fcqsp\""
Sep 11 12:22:43.315889 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.315768    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 12:22:43.315889 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.315795    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 12:22:43.315889 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.315824    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 12:22:43.319162 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.319134    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-8n2pn"]
Sep 11 12:22:43.413701 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.413663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6-webhook-cert\") pod \"maas-controller-7db788bf7f-8n2pn\" (UID: \"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6\") " pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.413701 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.413705    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6-metrics-certs\") pod \"maas-controller-7db788bf7f-8n2pn\" (UID: \"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6\") " pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.413932 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.413730    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5v54\" (UniqueName: \"kubernetes.io/projected/55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6-kube-api-access-k5v54\") pod \"maas-controller-7db788bf7f-8n2pn\" (UID: \"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6\") " pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.514850 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.514804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6-webhook-cert\") pod \"maas-controller-7db788bf7f-8n2pn\" (UID: \"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6\") " pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.514850 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.514851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6-metrics-certs\") pod \"maas-controller-7db788bf7f-8n2pn\" (UID: \"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6\") " pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.515087 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.514880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5v54\" (UniqueName: \"kubernetes.io/projected/55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6-kube-api-access-k5v54\") pod \"maas-controller-7db788bf7f-8n2pn\" (UID: \"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6\") " pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.517426 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.517394    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6-metrics-certs\") pod \"maas-controller-7db788bf7f-8n2pn\" (UID: \"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6\") " pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.517574 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.517401    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6-webhook-cert\") pod \"maas-controller-7db788bf7f-8n2pn\" (UID: \"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6\") " pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.522499 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.522472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5v54\" (UniqueName: \"kubernetes.io/projected/55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6-kube-api-access-k5v54\") pod \"maas-controller-7db788bf7f-8n2pn\" (UID: \"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6\") " pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.626848 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.626756    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:43.899077 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:43.898988    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7db788bf7f-8n2pn"]
Sep 11 12:22:43.902017 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:22:43.901987    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55d61eb3_6155_4bd8_9fa4_2fa7d7155cf6.slice/crio-35dc43b00998f87a20bec69ef5a4680259d54e9b0ac460736c38b4fb4ad571b4 WatchSource:0}: Error finding container 35dc43b00998f87a20bec69ef5a4680259d54e9b0ac460736c38b4fb4ad571b4: Status 404 returned error can't find the container with id 35dc43b00998f87a20bec69ef5a4680259d54e9b0ac460736c38b4fb4ad571b4
Sep 11 12:22:44.024741 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:44.024704    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7e2ef05-010b-407d-b74f-156dc8a8bfa9" path="/var/lib/kubelet/pods/e7e2ef05-010b-407d-b74f-156dc8a8bfa9/volumes"
Sep 11 12:22:44.465708 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:44.465666    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7db788bf7f-8n2pn" event={"ID":"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6","Type":"ContainerStarted","Data":"f24114177b90dc82c42303978012e25fe1a3a9d1e7101d3462c9314c772ec379"}
Sep 11 12:22:44.466075 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:44.465729    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7db788bf7f-8n2pn" event={"ID":"55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6","Type":"ContainerStarted","Data":"35dc43b00998f87a20bec69ef5a4680259d54e9b0ac460736c38b4fb4ad571b4"}
Sep 11 12:22:44.466075 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:44.465875    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:22:44.482852 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:44.482803    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7db788bf7f-8n2pn" podStartSLOduration=1.112258265 podStartE2EDuration="1.482787464s" totalImagesPullingTime="370.529199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:22:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:22:44.481230421 +0000 UTC m=+1872.981950700" watchObservedRunningTime="2026-09-11 12:22:44.482787464 +0000 UTC m=+1872.983507761"
Sep 11 12:22:55.472783 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:22:55.472703    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7db788bf7f-8n2pn"
Sep 11 12:26:44.760137 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.760100    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"]
Sep 11 12:26:44.764121 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.764098    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.774603 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.774573    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"]
Sep 11 12:26:44.852394 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.852355    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf9gm\" (UniqueName: \"kubernetes.io/projected/867446fc-bda5-4b0b-86dd-deba4fa11cca-kube-api-access-sf9gm\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.852394 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.852394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/867446fc-bda5-4b0b-86dd-deba4fa11cca-maas-api-tls\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.852625 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.852467    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/867446fc-bda5-4b0b-86dd-deba4fa11cca-openshift-service-ca\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.852625 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.852534    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/867446fc-bda5-4b0b-86dd-deba4fa11cca-metrics-certs\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.953998 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.953956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/867446fc-bda5-4b0b-86dd-deba4fa11cca-openshift-service-ca\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.954163 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.954038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/867446fc-bda5-4b0b-86dd-deba4fa11cca-metrics-certs\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.954163 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.954096    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf9gm\" (UniqueName: \"kubernetes.io/projected/867446fc-bda5-4b0b-86dd-deba4fa11cca-kube-api-access-sf9gm\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.954163 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.954124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/867446fc-bda5-4b0b-86dd-deba4fa11cca-maas-api-tls\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.954764 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.954740    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/867446fc-bda5-4b0b-86dd-deba4fa11cca-openshift-service-ca\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.956582 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.956559    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/867446fc-bda5-4b0b-86dd-deba4fa11cca-maas-api-tls\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.956732 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.956711    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/867446fc-bda5-4b0b-86dd-deba4fa11cca-metrics-certs\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:44.961994 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:44.961969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf9gm\" (UniqueName: \"kubernetes.io/projected/867446fc-bda5-4b0b-86dd-deba4fa11cca-kube-api-access-sf9gm\") pod \"maas-api-6f9765f64b-nkj44\" (UID: \"867446fc-bda5-4b0b-86dd-deba4fa11cca\") " pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:45.075713 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:45.075622    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:45.350611 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:45.350581    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"]
Sep 11 12:26:45.353547 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:26:45.353514    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod867446fc_bda5_4b0b_86dd_deba4fa11cca.slice/crio-7f07ce78096b401023f472de626b9ae697675610f307bf221c7bb925fdbdb26d WatchSource:0}: Error finding container 7f07ce78096b401023f472de626b9ae697675610f307bf221c7bb925fdbdb26d: Status 404 returned error can't find the container with id 7f07ce78096b401023f472de626b9ae697675610f307bf221c7bb925fdbdb26d
Sep 11 12:26:45.618147 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:45.618059    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44" event={"ID":"867446fc-bda5-4b0b-86dd-deba4fa11cca","Type":"ContainerStarted","Data":"7f07ce78096b401023f472de626b9ae697675610f307bf221c7bb925fdbdb26d"}
Sep 11 12:26:46.624632 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:46.624592    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44" event={"ID":"867446fc-bda5-4b0b-86dd-deba4fa11cca","Type":"ContainerStarted","Data":"5a2b596a9bfa4634a38584ee600f32e8349952186cd211cf0b34075f101d4312"}
Sep 11 12:26:46.625064 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:46.624781    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:26:46.641494 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:46.641442    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44" podStartSLOduration=2.148369265 podStartE2EDuration="2.641428638s" totalImagesPullingTime="493.059373ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:26:46.639832883 +0000 UTC m=+2115.140553152" watchObservedRunningTime="2026-09-11 12:26:46.641428638 +0000 UTC m=+2115.142148906"
Sep 11 12:26:52.635641 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:26:52.635608    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6f9765f64b-nkj44"
Sep 11 12:27:00.559491 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:00.559452    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"]
Sep 11 12:27:00.559973 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:00.559637    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd" podUID="d75989e8-0c4b-427b-9de7-c23cd9d5ce8e" containerName="maas-api" containerID="cri-o://cdd86b9918e01c3351ed1218985573e7e70dde4b1280bdac1c639b08475a11bf" gracePeriod=30
Sep 11 12:27:01.709199 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.709165    2562 generic.go:356] "Generic (PLEG): container finished" podID="d75989e8-0c4b-427b-9de7-c23cd9d5ce8e" containerID="cdd86b9918e01c3351ed1218985573e7e70dde4b1280bdac1c639b08475a11bf" exitCode=0
Sep 11 12:27:01.709592 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.709241    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd" event={"ID":"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e","Type":"ContainerDied","Data":"cdd86b9918e01c3351ed1218985573e7e70dde4b1280bdac1c639b08475a11bf"}
Sep 11 12:27:01.808087 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.808061    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:27:01.895971 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.895935    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-openshift-service-ca\") pod \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") "
Sep 11 12:27:01.896160 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.896042    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls\") pod \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") "
Sep 11 12:27:01.896160 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.896062    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-metrics-certs\") pod \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") "
Sep 11 12:27:01.896160 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.896079    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljlsz\" (UniqueName: \"kubernetes.io/projected/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-kube-api-access-ljlsz\") pod \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\" (UID: \"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e\") "
Sep 11 12:27:01.896364 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.896326    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e" (UID: "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:27:01.898275 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.898242    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e" (UID: "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:27:01.898418 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.898290    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e" (UID: "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:27:01.898418 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.898285    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-kube-api-access-ljlsz" (OuterVolumeSpecName: "kube-api-access-ljlsz") pod "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e" (UID: "d75989e8-0c4b-427b-9de7-c23cd9d5ce8e"). InnerVolumeSpecName "kube-api-access-ljlsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:27:01.997570 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.997528    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-maas-api-tls\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:27:01.997570 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.997561    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-metrics-certs\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:27:01.997570 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.997576    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljlsz\" (UniqueName: \"kubernetes.io/projected/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-kube-api-access-ljlsz\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:27:01.997829 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:01.997591    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e-openshift-service-ca\") on node \"ip-10-0-136-222.ec2.internal\" DevicePath \"\""
Sep 11 12:27:02.715364 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:02.715331    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"
Sep 11 12:27:02.715364 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:02.715355    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd" event={"ID":"d75989e8-0c4b-427b-9de7-c23cd9d5ce8e","Type":"ContainerDied","Data":"d3181b4c645bc6070c944386b099341841736e237e2d8edb23a99c182a2577c9"}
Sep 11 12:27:02.715832 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:02.715420    2562 scope.go:117] "RemoveContainer" containerID="cdd86b9918e01c3351ed1218985573e7e70dde4b1280bdac1c639b08475a11bf"
Sep 11 12:27:02.731884 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:02.731859    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"]
Sep 11 12:27:02.735154 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:02.735134    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-75bf587cb6-fprbd"]
Sep 11 12:27:04.025436 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:04.025393    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d75989e8-0c4b-427b-9de7-c23cd9d5ce8e" path="/var/lib/kubelet/pods/d75989e8-0c4b-427b-9de7-c23cd9d5ce8e/volumes"
Sep 11 12:27:25.187667 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:25.187585    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7db788bf7f-8n2pn_55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6/manager/0.log"
Sep 11 12:27:25.668950 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:25.668861    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j_e9f7f12a-f7ee-4702-8dfe-22145fb7939a/copy-manifests/0.log"
Sep 11 12:27:25.675783 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:25.675752    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j_e9f7f12a-f7ee-4702-8dfe-22145fb7939a/manager/0.log"
Sep 11 12:27:26.159517 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:26.159484    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6f9765f64b-nkj44_867446fc-bda5-4b0b-86dd-deba4fa11cca/maas-api/0.log"
Sep 11 12:27:26.497687 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:26.497656    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-fghx7_f3673b99-ed29-4c4b-b895-2ffaa23d43ca/postgres/0.log"
Sep 11 12:27:28.598956 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:28.598916    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cqzqm_672739d3-4ea1-4d9a-8184-3c87701a061f/discovery/0.log"
Sep 11 12:27:29.263624 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:29.263594    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4mnp2_e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d/payload-processing/0.log"
Sep 11 12:27:29.866147 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:29.866112    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-dg8qb_caabba6f-86d9-457f-9b19-5507e9eae073/main/0.log"
Sep 11 12:27:30.113913 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:30.113882    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2f7dm_2e9a7825-ad1c-42c6-ace4-b07c0985276e/main/0.log"
Sep 11 12:27:39.039083 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:39.039049    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fqcqr_e693b077-566f-41ce-b702-f9ea0084ed60/global-pull-secret-syncer/0.log"
Sep 11 12:27:39.200431 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:39.200388    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6f2ph_a24a34f9-b1e8-4418-8f31-3bebcf7dcbfd/konnectivity-agent/0.log"
Sep 11 12:27:39.289012 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:39.288981    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-222.ec2.internal_0b320866b4a982a72db4f26f5aaaff86/haproxy/0.log"
Sep 11 12:27:45.486780 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:45.486748    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc4340a-85dd-491b-8ac1-b080841d6ca0/alertmanager/0.log"
Sep 11 12:27:45.509983 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:45.509945    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc4340a-85dd-491b-8ac1-b080841d6ca0/config-reloader/0.log"
Sep 11 12:27:45.532194 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:45.532163    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc4340a-85dd-491b-8ac1-b080841d6ca0/kube-rbac-proxy-web/0.log"
Sep 11 12:27:45.553296 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:45.553273    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc4340a-85dd-491b-8ac1-b080841d6ca0/kube-rbac-proxy/0.log"
Sep 11 12:27:45.576500 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:45.576473    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc4340a-85dd-491b-8ac1-b080841d6ca0/kube-rbac-proxy-metric/0.log"
Sep 11 12:27:45.597844 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:45.597818    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc4340a-85dd-491b-8ac1-b080841d6ca0/prom-label-proxy/0.log"
Sep 11 12:27:45.623283 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:45.623259    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8cc4340a-85dd-491b-8ac1-b080841d6ca0/init-config-reloader/0.log"
Sep 11 12:27:45.863901 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:45.863813    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7xdvd_7109f4b4-3483-45f3-9bbd-650fc0ca8280/node-exporter/0.log"
Sep 11 12:27:45.883982 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:45.883945    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7xdvd_7109f4b4-3483-45f3-9bbd-650fc0ca8280/kube-rbac-proxy/0.log"
Sep 11 12:27:45.904436 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:45.904409    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7xdvd_7109f4b4-3483-45f3-9bbd-650fc0ca8280/init-textfile/0.log"
Sep 11 12:27:46.088467 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.088435    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-dlgkn_a2dcce9c-c706-4269-baf4-f1a153b9345d/kube-rbac-proxy-main/0.log"
Sep 11 12:27:46.109402 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.109368    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-dlgkn_a2dcce9c-c706-4269-baf4-f1a153b9345d/kube-rbac-proxy-self/0.log"
Sep 11 12:27:46.131484 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.131418    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-dlgkn_a2dcce9c-c706-4269-baf4-f1a153b9345d/openshift-state-metrics/0.log"
Sep 11 12:27:46.359768 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.359733    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-lwzhv_5026325a-81a2-44b8-a246-2d650d5852b7/prometheus-operator/0.log"
Sep 11 12:27:46.377129 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.377109    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-lwzhv_5026325a-81a2-44b8-a246-2d650d5852b7/kube-rbac-proxy/0.log"
Sep 11 12:27:46.404665 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.404593    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-kppjz_5bf9ef2a-e8b5-49f8-8e5f-658d828e1a44/prometheus-operator-admission-webhook/0.log"
Sep 11 12:27:46.436860 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.436831    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77c496dd8f-9k224_c177b06d-d37c-4e70-a22b-4646b90d4e4b/telemeter-client/0.log"
Sep 11 12:27:46.460255 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.460230    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77c496dd8f-9k224_c177b06d-d37c-4e70-a22b-4646b90d4e4b/reload/0.log"
Sep 11 12:27:46.504417 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.504393    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77c496dd8f-9k224_c177b06d-d37c-4e70-a22b-4646b90d4e4b/kube-rbac-proxy/0.log"
Sep 11 12:27:46.560189 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.560163    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68b56d7dcb-4slw2_b8276f62-610e-4721-b276-1f648166036e/thanos-query/0.log"
Sep 11 12:27:46.583898 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.583870    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68b56d7dcb-4slw2_b8276f62-610e-4721-b276-1f648166036e/kube-rbac-proxy-web/0.log"
Sep 11 12:27:46.618673 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.618637    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68b56d7dcb-4slw2_b8276f62-610e-4721-b276-1f648166036e/kube-rbac-proxy/0.log"
Sep 11 12:27:46.635141 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.635111    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68b56d7dcb-4slw2_b8276f62-610e-4721-b276-1f648166036e/prom-label-proxy/0.log"
Sep 11 12:27:46.661035 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.660961    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68b56d7dcb-4slw2_b8276f62-610e-4721-b276-1f648166036e/kube-rbac-proxy-rules/0.log"
Sep 11 12:27:46.687853 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:46.687822    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68b56d7dcb-4slw2_b8276f62-610e-4721-b276-1f648166036e/kube-rbac-proxy-metrics/0.log"
Sep 11 12:27:48.360343 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:48.360248    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-zq4mw_b669f014-1829-47a8-9d3a-bbb01d4d6eb6/networking-console-plugin/0.log"
Sep 11 12:27:49.771826 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.771790    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"]
Sep 11 12:27:49.772203 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.772183    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d75989e8-0c4b-427b-9de7-c23cd9d5ce8e" containerName="maas-api"
Sep 11 12:27:49.772203 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.772195    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d75989e8-0c4b-427b-9de7-c23cd9d5ce8e" containerName="maas-api"
Sep 11 12:27:49.772280 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.772265    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d75989e8-0c4b-427b-9de7-c23cd9d5ce8e" containerName="maas-api"
Sep 11 12:27:49.775561 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.775540    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.777920 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.777866    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rhtnl\"/\"openshift-service-ca.crt\""
Sep 11 12:27:49.778044 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.777976    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rhtnl\"/\"default-dockercfg-blx5s\""
Sep 11 12:27:49.778585 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.778569    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rhtnl\"/\"kube-root-ca.crt\""
Sep 11 12:27:49.789299 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.789276    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"]
Sep 11 12:27:49.847859 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.847829    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-lib-modules\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.848037 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.847893    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-podres\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.848037 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.847964    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkrj7\" (UniqueName: \"kubernetes.io/projected/834eb554-ac8d-4e4c-9811-63563a5d919c-kube-api-access-pkrj7\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.848037 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.847998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-sys\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.848037 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.848033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-proc\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.949138 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.949086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-lib-modules\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.949349 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.949200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-podres\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.949349 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.949246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkrj7\" (UniqueName: \"kubernetes.io/projected/834eb554-ac8d-4e4c-9811-63563a5d919c-kube-api-access-pkrj7\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.949349 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.949294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-lib-modules\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.949544 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.949375    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-podres\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.949544 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.949375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-sys\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.949544 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.949436    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-sys\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.949544 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.949452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-proc\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.949544 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.949496    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/834eb554-ac8d-4e4c-9811-63563a5d919c-proc\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:49.958118 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:49.958087    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkrj7\" (UniqueName: \"kubernetes.io/projected/834eb554-ac8d-4e4c-9811-63563a5d919c-kube-api-access-pkrj7\") pod \"perf-node-gather-daemonset-fkk2x\" (UID: \"834eb554-ac8d-4e4c-9811-63563a5d919c\") " pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:50.085722 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:50.085636    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:50.359205 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:50.359143    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"]
Sep 11 12:27:50.362199 ip-10-0-136-222 kubenswrapper[2562]: W0911 12:27:50.362169    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod834eb554_ac8d_4e4c_9811_63563a5d919c.slice/crio-96add0aa017e00fbc3155cf359a7dc3805df2d61e5d046baaa7d70e942850e43 WatchSource:0}: Error finding container 96add0aa017e00fbc3155cf359a7dc3805df2d61e5d046baaa7d70e942850e43: Status 404 returned error can't find the container with id 96add0aa017e00fbc3155cf359a7dc3805df2d61e5d046baaa7d70e942850e43
Sep 11 12:27:50.940697 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:50.940660    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x" event={"ID":"834eb554-ac8d-4e4c-9811-63563a5d919c","Type":"ContainerStarted","Data":"f8c533fea49137071030ad8ff0a0eaeaebe577108738caf0bf2f3d95f24ecfb5"}
Sep 11 12:27:50.941141 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:50.940713    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x" event={"ID":"834eb554-ac8d-4e4c-9811-63563a5d919c","Type":"ContainerStarted","Data":"96add0aa017e00fbc3155cf359a7dc3805df2d61e5d046baaa7d70e942850e43"}
Sep 11 12:27:50.941141 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:50.940808    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:50.955066 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:50.954957    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x" podStartSLOduration=1.954936646 podStartE2EDuration="1.954936646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:27:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:27:50.954890226 +0000 UTC m=+2179.455610496" watchObservedRunningTime="2026-09-11 12:27:50.954936646 +0000 UTC m=+2179.455656913"
Sep 11 12:27:51.583547 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:51.583512    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jsxzt_28c2284e-89be-4c9f-a804-e1a2373a8f92/dns/0.log"
Sep 11 12:27:51.604757 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:51.604724    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jsxzt_28c2284e-89be-4c9f-a804-e1a2373a8f92/kube-rbac-proxy/0.log"
Sep 11 12:27:51.725594 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:51.725561    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xtfmj_86e098b8-e73f-43c1-bfe3-242bd950b4bd/dns-node-resolver/0.log"
Sep 11 12:27:52.287738 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:52.287699    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6f75696d6b-4sscr_cd59708e-fba7-4384-9bbb-c0147c65eb81/registry/0.log"
Sep 11 12:27:52.334384 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:52.334341    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lv7nr_1e028310-ec98-4645-ad5c-48db2103260d/node-ca/0.log"
Sep 11 12:27:53.709431 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:53.709395    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cqzqm_672739d3-4ea1-4d9a-8184-3c87701a061f/discovery/0.log"
Sep 11 12:27:53.895205 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:53.895174    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4mnp2_e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d/payload-processing/0.log"
Sep 11 12:27:54.819326 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:54.819282    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d5cxg_1d546130-4201-41ee-ba52-d2f1d1b9e205/serve-healthcheck-canary/0.log"
Sep 11 12:27:55.286110 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:55.286058    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-znzgz_09200696-2c09-4c98-8b6e-6d3116c2319a/insights-operator/0.log"
Sep 11 12:27:55.286673 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:55.286656    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-znzgz_09200696-2c09-4c98-8b6e-6d3116c2319a/insights-operator/1.log"
Sep 11 12:27:55.392361 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:55.392330    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bw87x_0c6f3f60-4cd7-4811-83d0-5d3af4659599/kube-rbac-proxy/0.log"
Sep 11 12:27:55.419765 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:55.419738    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bw87x_0c6f3f60-4cd7-4811-83d0-5d3af4659599/exporter/0.log"
Sep 11 12:27:55.442608 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:55.442575    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bw87x_0c6f3f60-4cd7-4811-83d0-5d3af4659599/extractor/0.log"
Sep 11 12:27:56.956287 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:56.956259    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rhtnl/perf-node-gather-daemonset-fkk2x"
Sep 11 12:27:58.052804 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:58.052768    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7db788bf7f-8n2pn_55d61eb3-6155-4bd8-9fa4-2fa7d7155cf6/manager/0.log"
Sep 11 12:27:58.284084 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:58.284053    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j_e9f7f12a-f7ee-4702-8dfe-22145fb7939a/manager/0.log"
Sep 11 12:27:58.307804 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:58.307777    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cbb9c7bc-n2k8j_e9f7f12a-f7ee-4702-8dfe-22145fb7939a/copy-manifests/0.log"
Sep 11 12:27:59.914238 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:27:59.914198    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-56c7cddbd4-lctbg_104e6446-c5d4-42d8-b32a-434f2ca4a043/manager/0.log"
Sep 11 12:28:00.630988 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:00.630952    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cqzqm_672739d3-4ea1-4d9a-8184-3c87701a061f/discovery/0.log"
Sep 11 12:28:00.767569 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:00.767533    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4mnp2_e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d/payload-processing/0.log"
Sep 11 12:28:02.764265 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:02.764225    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cqzqm_672739d3-4ea1-4d9a-8184-3c87701a061f/discovery/0.log"
Sep 11 12:28:02.928885 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:02.928847    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4mnp2_e884b1d0-9c9b-4f91-86cc-d2d0a90dc70d/payload-processing/0.log"
Sep 11 12:28:04.828394 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:04.828365    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-5vlzm_b9f83fe6-6ab6-4718-b9f5-496ce4992286/migrator/0.log"
Sep 11 12:28:04.849397 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:04.849366    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-5vlzm_b9f83fe6-6ab6-4718-b9f5-496ce4992286/graceful-termination/0.log"
Sep 11 12:28:05.186933 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:05.186903    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-wqtf9_a9479947-3863-4fcd-8b58-ded0146677a9/kube-storage-version-migrator-operator/1.log"
Sep 11 12:28:05.188726 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:05.188704    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-wqtf9_a9479947-3863-4fcd-8b58-ded0146677a9/kube-storage-version-migrator-operator/0.log"
Sep 11 12:28:06.454678 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:06.454645    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkpgh_7c05f73e-008f-497c-9e03-6b411a683a03/kube-multus-additional-cni-plugins/0.log"
Sep 11 12:28:06.476322 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:06.476278    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkpgh_7c05f73e-008f-497c-9e03-6b411a683a03/egress-router-binary-copy/0.log"
Sep 11 12:28:06.496701 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:06.496676    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkpgh_7c05f73e-008f-497c-9e03-6b411a683a03/cni-plugins/0.log"
Sep 11 12:28:06.518812 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:06.518786    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkpgh_7c05f73e-008f-497c-9e03-6b411a683a03/bond-cni-plugin/0.log"
Sep 11 12:28:06.543151 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:06.543119    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkpgh_7c05f73e-008f-497c-9e03-6b411a683a03/routeoverride-cni/0.log"
Sep 11 12:28:06.564997 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:06.564971    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkpgh_7c05f73e-008f-497c-9e03-6b411a683a03/whereabouts-cni-bincopy/0.log"
Sep 11 12:28:06.588123 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:06.588099    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkpgh_7c05f73e-008f-497c-9e03-6b411a683a03/whereabouts-cni/0.log"
Sep 11 12:28:06.688261 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:06.688225    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pxcxn_2970bc44-0b6c-4fc7-89d3-c5db6b9d10c6/kube-multus/0.log"
Sep 11 12:28:06.873671 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:06.873586    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w98wm_142371cb-79c5-44b6-9e7d-9c01359b29f4/network-metrics-daemon/0.log"
Sep 11 12:28:06.890209 ip-10-0-136-222 kubenswrapper[2562]: I0911 12:28:06.890179    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w98wm_142371cb-79c5-44b6-9e7d-9c01359b29f4/kube-rbac-proxy/0.log"