Sep 11 12:45:10.393828 ip-10-0-136-55 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 12:45:10.393839 ip-10-0-136-55 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 12:45:10.393846 ip-10-0-136-55 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 12:45:10.394044 ip-10-0-136-55 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 12:45:20.497625 ip-10-0-136-55 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 12:45:20.497641 ip-10-0-136-55 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7f7a9d43152d4db3bed151de45e9df71 --
Sep 11 12:47:11.819152 ip-10-0-136-55 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 12:47:12.231077 ip-10-0-136-55 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 12:47:12.231077 ip-10-0-136-55 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 12:47:12.231077 ip-10-0-136-55 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 12:47:12.231077 ip-10-0-136-55 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 12:47:12.231077 ip-10-0-136-55 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 12:47:12.232112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.232014    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 12:47:12.237197 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237167    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:47:12.237197 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237190    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:47:12.237197 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237197    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:47:12.237197 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237203    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:47:12.237197 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237209    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237214    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237218    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237222    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237226    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237230    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237234    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237238    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237241    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237244    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237247    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237251    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237255    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237260    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237265    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237268    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237271    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237275    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237279    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237282    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:47:12.237564 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237286    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237289    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237294    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237298    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237301    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237306    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237312    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237318    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237322    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237325    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237330    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237334    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237338    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237342    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237349    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237354    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237358    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237363    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237367    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237372    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:47:12.238329 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237376    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237380    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237383    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237388    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237392    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237396    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237400    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237404    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237408    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237416    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237421    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237426    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237431    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237435    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237439    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237444    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237448    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237453    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237457    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:47:12.238847 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237461    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237465    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237469    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237474    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237478    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237482    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237486    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237492    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237496    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237501    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237506    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237512    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237516    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237521    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237526    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237530    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237535    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237539    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237545    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237549    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:47:12.239308 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237553    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237557    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237561    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237565    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237569    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237573    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237578    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237581    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237586    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237590    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237594    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237598    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237602    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237606    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237611    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.237615    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238679    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238691    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238695    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238698    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:47:12.239824 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238701    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238704    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238706    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238709    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238713    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238715    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238718    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238721    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238724    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238727    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238730    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238732    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238735    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238737    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238740    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238742    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238745    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238748    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238750    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238753    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:47:12.240311 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238756    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238758    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238761    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238763    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238785    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238789    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238792    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238795    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238799    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238802    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238805    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238808    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238812    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238816    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238820    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238823    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238826    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238828    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238831    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238833    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:47:12.241141 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238836    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238838    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238841    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238843    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238846    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238849    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238851    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238854    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238857    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238860    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238862    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238866    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238868    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238871    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238873    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238876    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238878    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238881    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238884    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238886    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:47:12.241916 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238895    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238897    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238902    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238905    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238908    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238911    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238914    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238916    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238919    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238921    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238924    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238927    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238930    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238933    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238935    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238939    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238944    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238947    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238951    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238953    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:47:12.242720 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238956    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238959    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238962    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238965    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238967    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238970    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238972    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238975    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238978    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238980    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238982    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238986    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238992    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238995    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.238998    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239399    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239404    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239408    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239412    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:47:12.243246 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239416    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239419    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239422    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239426    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239429    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239431    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239434    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239437    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239439    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239442    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239444    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239447    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239449    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239452    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239455    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239457    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239460    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239462    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239465    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239467    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:47:12.243744 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239470    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239473    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239475    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239478    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239480    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239486    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239488    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239491    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239494    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239497    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239500    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239502    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239507    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239510    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239513    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239516    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239520    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239523    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239526    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:47:12.244265 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239529    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239531    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239534    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239537    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239539    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239541    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239544    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239546    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239549    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239551    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239554    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239556    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239558    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239561    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239564    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239566    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239569    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239571    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239577    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239579    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:47:12.244771 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239582    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239584    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239587    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239590    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239592    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239595    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239597    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239600    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239602    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239605    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239607    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239610    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239612    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239616    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239619    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239621    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239624    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239626    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239629    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239632    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:47:12.245287 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239635    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239652    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239656    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239658    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239661    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239664    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239666    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239669    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239671    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239674    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239679    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239682    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239684    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239687    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239690    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.239693    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239770    2564 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239812    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239819    2564 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239824    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239828    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 12:47:12.245795 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239832    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239836    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239841    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239844    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239847    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239851    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239855    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239858    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239861    2564 flags.go:64] FLAG: --cgroup-root=""
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239864    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239867    2564 flags.go:64] FLAG: --client-ca-file=""
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239870    2564 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239873    2564 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239877    2564 flags.go:64] FLAG: --cluster-domain=""
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239880    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239883    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239886    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239890    2564 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239894    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239897    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239900    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239905    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239908    2564 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239911    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 12:47:12.246327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239915    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239918    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239921    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239926    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239929    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239932    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239935    2564 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239938    2564 flags.go:64] FLAG: --enable-server="true"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239941    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239946    2564 flags.go:64] FLAG: --event-burst="100"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239949    2564 flags.go:64] FLAG: --event-qps="50"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239952    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239956    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239958    2564 flags.go:64] FLAG: --eviction-hard=""
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239962    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239965    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239968    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239971    2564 flags.go:64] FLAG: --eviction-soft=""
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239974    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239977    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239980    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239983    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239986    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239988    2564 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239991    2564 flags.go:64] FLAG: --feature-gates=""
Sep 11 12:47:12.246927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239995    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.239998    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240001    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240004    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240008    2564 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240011    2564 flags.go:64] FLAG: --help="false"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240015    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240018    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240021    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240024    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240027    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240032    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240035    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240038    2564 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240041    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240044    2564 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240047    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240050    2564 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240053    2564 flags.go:64] FLAG: --kube-reserved=""
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240056    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240058    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240061    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240064    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240067    2564 flags.go:64] FLAG: --lock-file=""
Sep 11 12:47:12.247545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240070    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240073    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240076    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240082    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240085    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240088    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240090    2564 flags.go:64] FLAG: --logging-format="text"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240094    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240098    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240101    2564 flags.go:64] FLAG: --manifest-url=""
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240103    2564 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240108    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240112    2564 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240116    2564 flags.go:64] FLAG: --max-pods="110"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240119    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240122    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240125    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240129    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240132    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240135    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240138    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240146    2564 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240149    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240153    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 12:47:12.248170 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240156    2564 flags.go:64] FLAG: --pod-cidr=""
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240159    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240165    2564 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240168    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240172    2564 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240175    2564 flags.go:64] FLAG: --port="10250"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240178    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240181    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05f37f987c5e376b7"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240184    2564 flags.go:64] FLAG: --qos-reserved=""
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240187    2564 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240191    2564 flags.go:64] FLAG: --register-node="true"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240193    2564 flags.go:64] FLAG: --register-with-taints=""
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240198    2564 flags.go:64] FLAG: --registry-burst="10"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240200    2564 flags.go:64] FLAG: --registry-qps="5"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240203    2564 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240206    2564 flags.go:64] FLAG: --reserved-memory=""
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240210    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240213    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240216    2564 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240219    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240222    2564 flags.go:64] FLAG: --runonce="false"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240225    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240228    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240231    2564 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240234    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 12:47:12.249092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240237    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240240    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240243    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240246    2564 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240249    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240252    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240255    2564 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240258    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240261    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240264    2564 flags.go:64] FLAG: --system-cgroups=""
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240266    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240272    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240275    2564 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240277    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240282    2564 flags.go:64] FLAG: --tls-min-version=""
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240285    2564 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240287    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240297    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240302    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240306    2564 flags.go:64] FLAG: --v="2"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240310    2564 flags.go:64] FLAG: --version="false"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240314    2564 flags.go:64] FLAG: --vmodule=""
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240318    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.240321    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240869    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:47:12.250181 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240889    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240903    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240911    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240924    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240931    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240946    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240955    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240962    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240969    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240976    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240984    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.240991    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241000    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241007    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241014    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241020    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241025    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241030    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241039    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241044    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:47:12.251193 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241048    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241054    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241058    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241063    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241067    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241071    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241075    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241080    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241085    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241089    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241098    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241103    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241107    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241113    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241118    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241124    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241128    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241133    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241138    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241143    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:47:12.251716 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241148    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241153    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241162    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241166    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241171    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241176    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241180    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241185    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241189    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241193    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241197    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241202    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241206    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241210    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241214    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241223    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241227    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241231    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241235    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241242    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:47:12.252581 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241246    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241250    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241254    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241258    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241262    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241269    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241273    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241280    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241291    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241298    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241304    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241309    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241316    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241322    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241326    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241331    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241335    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241340    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241345    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:47:12.253465 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241354    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241358    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241362    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241367    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241371    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241375    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241380    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241384    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241389    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241393    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241397    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241401    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241406    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241415    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241419    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241423    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241428    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241432    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:47:12.254078 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.241437    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.242395    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.251009    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.251035    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.251764    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.251778    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251878    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251889    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251894    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251899    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251903    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251908    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251912    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251916    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251921    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:47:12.254877 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251925    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251929    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251934    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251938    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251942    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251947    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251951    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251955    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251959    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251965    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251971    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251977    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251982    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251988    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251993    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.251997    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252001    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252005    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252010    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:47:12.255480 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252015    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252020    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252024    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252028    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252032    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252037    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252041    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252046    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252050    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252054    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252059    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252063    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252067    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252071    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252075    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252080    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252084    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252089    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252093    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252097    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:47:12.256011 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252101    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252105    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252110    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252114    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252118    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252122    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252127    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252132    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252136    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252140    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252144    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252148    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252153    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252158    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252162    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252166    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252171    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252175    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252179    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252184    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:47:12.256626 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252188    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252192    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252196    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252200    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252205    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252210    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252214    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252218    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252222    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252226    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252230    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252234    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252238    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252243    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252248    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252252    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252256    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252261    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252265    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:47:12.257175 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252268    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252273    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252277    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252281    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252286    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252298    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252303    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252310    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252316    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252321    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252328    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252335    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.252343    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252540    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:47:12.257700 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252548    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252553    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252558    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252562    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252566    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252571    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252575    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252579    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252583    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252589    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252595    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252599    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252604    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252608    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252612    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252617    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252621    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252627    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:47:12.258084 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252633    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252654    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252659    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252663    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252668    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252672    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252676    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252680    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252683    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252687    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252692    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252696    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252701    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252705    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252709    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252713    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252717    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252721    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252725    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252729    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:47:12.258525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252733    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252737    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252741    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252746    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252749    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252754    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252758    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252762    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252766    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252771    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252776    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252780    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252784    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252788    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252792    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252797    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252801    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252806    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252811    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252815    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:47:12.259164 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252819    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252824    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252828    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252832    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252837    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252842    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252847    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252851    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252855    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252859    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252864    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252868    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252872    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252877    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252881    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252886    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252890    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252895    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252898    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252903    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:47:12.259718 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252907    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252911    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252916    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252920    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252926    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252932    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252937    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252941    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252946    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252950    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252954    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252959    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252963    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252967    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252972    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252976    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252980    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252984    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252990    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:47:12.260268 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:12.252995    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:47:12.261056 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.253003    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 12:47:12.261056 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.253202    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 12:47:12.261056 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.256030    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 12:47:12.261056 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.257143    2564 server.go:1019] "Starting client certificate rotation"
Sep 11 12:47:12.261056 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.257258    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 12:47:12.261056 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.257331    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 12:47:12.261056 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.260854    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 12:47:12.262314 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.262283    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 12:47:12.270880 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.270856    2564 log.go:25] "Validated CRI v1 runtime API"
Sep 11 12:47:12.275853 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.275829    2564 log.go:25] "Validated CRI v1 image API"
Sep 11 12:47:12.279088 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.279060    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 12:47:12.281616 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.281591    2564 fs.go:135] Filesystem UUIDs: map[56e385fb-73b6-47a6-9e4d-979063068657:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 95e5b3c0-c52d-4c09-bbdf-8fe38f589361:/dev/nvme0n1p4]
Sep 11 12:47:12.281616 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.281612    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 12:47:12.287238 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.287214    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 12:47:12.287690 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.287563    2564 manager.go:216] Machine: {Timestamp:2026-09-11 12:47:12.28606478 +0000 UTC m=+0.348371728 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098580 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bae4fd95c2224f4decc5e3978c34c SystemUUID:ec2bae4f-d95c-2224-f4de-cc5e3978c34c BootID:7f7a9d43-152d-4db3-bed1-51de45e9df71 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:da:22:8b:4f:7d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:da:22:8b:4f:7d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:ec:bf:aa:13:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 12:47:12.287690 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.287685    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 12:47:12.287831 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.287819    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 12:47:12.291458 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.291411    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 12:47:12.291669 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.291462    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-55.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 12:47:12.292279 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.292266    2564 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 12:47:12.292318 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.292284    2564 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 12:47:12.292318 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.292300    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 12:47:12.293403 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.293388    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 12:47:12.326378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.326352    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6krm7"
Sep 11 12:47:12.333685 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.333654    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6krm7"
Sep 11 12:47:12.336816 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.336788    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 12:47:12.343172 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.343147    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 12:47:12.343299 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.343292    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 12:47:12.343334 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.343324    2564 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 12:47:12.343366 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.343335    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 12:47:12.343366 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.343354    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 12:47:12.343366 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.343363    2564 kubelet.go:398] "Adding apiserver pod source"
Sep 11 12:47:12.343443 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.343373    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 12:47:12.343848 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.343826    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 12:47:12.343848 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.343852    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 12:47:12.346163 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.346139    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 12:47:12.348259 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.348243    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 12:47:12.348347 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.348273    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 12:47:12.349895 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349868    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 12:47:12.349895 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349894    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 12:47:12.350023 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349903    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 12:47:12.350023 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349912    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 12:47:12.350023 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349925    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 12:47:12.350023 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349933    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 12:47:12.350023 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349953    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 12:47:12.350023 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349961    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 12:47:12.350023 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349972    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 12:47:12.350023 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349981    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 12:47:12.350023 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.349993    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 12:47:12.350023 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.350002    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 12:47:12.351636 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.351619    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 12:47:12.351636 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.351634    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 12:47:12.353226 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.353208    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:47:12.356882 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.356841    2564 server.go:1268] "Started kubelet"
Sep 11 12:47:12.357152 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.357069    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 12:47:12.357266 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.357234    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 12:47:12.357444 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.357346    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 12:47:12.358026 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.358008    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:47:12.358280 ip-10-0-136-55 systemd[1]: Started Kubernetes Kubelet.
Sep 11 12:47:12.358589 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.358553    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 12:47:12.359114 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.359091    2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-55.ec2.internal" not found
Sep 11 12:47:12.360459 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.360443    2564 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 12:47:12.362586 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:12.362543    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 12:47:12.367225 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.367203    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 12:47:12.367877 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.367855    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 12:47:12.368039 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:12.368018    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-55.ec2.internal\" not found"
Sep 11 12:47:12.368574 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.368556    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 12:47:12.368574 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.368557    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 12:47:12.368708 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.368581    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 12:47:12.368708 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.368679    2564 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 12:47:12.368708 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.368687    2564 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 12:47:12.369482 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.369467    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:47:12.370134 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.370116    2564 factory.go:55] Registering systemd factory
Sep 11 12:47:12.370134 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.370135    2564 factory.go:223] Registration of the systemd container factory successfully
Sep 11 12:47:12.370364 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.370349    2564 factory.go:153] Registering CRI-O factory
Sep 11 12:47:12.370364 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.370364    2564 factory.go:223] Registration of the crio container factory successfully
Sep 11 12:47:12.370485 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.370411    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 12:47:12.370485 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.370434    2564 factory.go:103] Registering Raw factory
Sep 11 12:47:12.370485 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.370468    2564 manager.go:1196] Started watching for new ooms in manager
Sep 11 12:47:12.370963 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.370941    2564 manager.go:349] Starting recovery of all containers
Sep 11 12:47:12.371702 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:12.371675    2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-55.ec2.internal\" not found" node="ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.372801 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.372778    2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-55.ec2.internal" not found
Sep 11 12:47:12.383072 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.383053    2564 manager.go:354] Recovery completed
Sep 11 12:47:12.387857 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.387839    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 12:47:12.389839 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.389820    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-55.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 12:47:12.389917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.389854    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-55.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 12:47:12.389917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.389867    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-55.ec2.internal" event="NodeHasSufficientPID"
Sep 11 12:47:12.390436 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.390423    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 12:47:12.390436 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.390432    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 12:47:12.390546 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.390450    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 12:47:12.390546 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.390503    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 12:47:12.391240 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.391222    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-55.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 12:47:12.391288 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.391249    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-55.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 12:47:12.391288 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.391263    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-55.ec2.internal" event="NodeHasSufficientPID"
Sep 11 12:47:12.392743 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.392731    2564 policy_none.go:49] "None policy: Start"
Sep 11 12:47:12.392788 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.392748    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 12:47:12.392788 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.392759    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 12:47:12.394761 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.394749    2564 policy_none.go:47] "Start"
Sep 11 12:47:12.429734 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.429703    2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-55.ec2.internal" not found
Sep 11 12:47:12.449260 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.431333    2564 manager.go:337] "Starting Device Plugin manager"
Sep 11 12:47:12.449260 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:12.431373    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 12:47:12.449260 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.431384    2564 server.go:87] "Starting device plugin registration server"
Sep 11 12:47:12.449260 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.431667    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 12:47:12.449260 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.431678    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 12:47:12.449260 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.431778    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 12:47:12.449260 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.431870    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 12:47:12.449260 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.431879    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 12:47:12.449260 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:12.432362    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 12:47:12.449260 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:12.432405    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-55.ec2.internal\" not found"
Sep 11 12:47:12.475274 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.475240    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 12:47:12.476488 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.476461    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 12:47:12.476672 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.476518    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 12:47:12.476672 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.476541    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 12:47:12.476672 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:12.476584    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 12:47:12.478586 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.478567    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:47:12.532692 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.532563    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 12:47:12.533601 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.533579    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-55.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 12:47:12.533753 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.533614    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-55.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 12:47:12.533753 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.533626    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-55.ec2.internal" event="NodeHasSufficientPID"
Sep 11 12:47:12.533753 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.533669    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.544341 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.544308    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.576932 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.576865    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal"]
Sep 11 12:47:12.580148 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.580124    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.582325 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.582302    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.597351 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.597328    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.601555 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.601537    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.611042 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.611018    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 12:47:12.611245 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.611230    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 12:47:12.670303 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.670271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0dbe8941313665e9751104dfb30bc72-config\") pod \"kube-apiserver-proxy-ip-10-0-136-55.ec2.internal\" (UID: \"f0dbe8941313665e9751104dfb30bc72\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.670303 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.670304    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c21e6ec17475096785af4f25e5a0af6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal\" (UID: \"1c21e6ec17475096785af4f25e5a0af6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.670512 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.670325    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c21e6ec17475096785af4f25e5a0af6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal\" (UID: \"1c21e6ec17475096785af4f25e5a0af6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.770855 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.770815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0dbe8941313665e9751104dfb30bc72-config\") pod \"kube-apiserver-proxy-ip-10-0-136-55.ec2.internal\" (UID: \"f0dbe8941313665e9751104dfb30bc72\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.770855 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.770850    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c21e6ec17475096785af4f25e5a0af6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal\" (UID: \"1c21e6ec17475096785af4f25e5a0af6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.771055 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.770878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c21e6ec17475096785af4f25e5a0af6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal\" (UID: \"1c21e6ec17475096785af4f25e5a0af6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.771055 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.770940    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0dbe8941313665e9751104dfb30bc72-config\") pod \"kube-apiserver-proxy-ip-10-0-136-55.ec2.internal\" (UID: \"f0dbe8941313665e9751104dfb30bc72\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.771055 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.770941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c21e6ec17475096785af4f25e5a0af6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal\" (UID: \"1c21e6ec17475096785af4f25e5a0af6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.771055 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.770948    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c21e6ec17475096785af4f25e5a0af6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal\" (UID: \"1c21e6ec17475096785af4f25e5a0af6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.914218 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.914119    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:12.915242 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:12.915224    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal"
Sep 11 12:47:13.257460 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.257225    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 12:47:13.257923 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.257551    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 12:47:13.257923 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.257594    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 12:47:13.257923 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.257603    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 12:47:13.342282 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.342186    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 12:42:13 +0000 UTC" deadline="2028-05-09 02:16:40.477268088 +0000 UTC"
Sep 11 12:47:13.342282 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.342278    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14533h29m27.134993923s"
Sep 11 12:47:13.344359 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.344341    2564 apiserver.go:52] "Watching apiserver"
Sep 11 12:47:13.351559 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.351535    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 12:47:13.353068 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.353041    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh","openshift-image-registry/node-ca-zbccg","openshift-multus/network-metrics-daemon-4f7x7","openshift-network-operator/iptables-alerter-rkngq","openshift-ovn-kubernetes/ovnkube-node-dscjx","kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal","openshift-cluster-node-tuning-operator/tuned-rx64q","openshift-dns/node-resolver-7lzcr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal","openshift-multus/multus-additional-cni-plugins-jsq92","openshift-multus/multus-tv9wd","openshift-network-diagnostics/network-check-target-pdqx9","kube-system/global-pull-secret-syncer-2btsm","kube-system/konnectivity-agent-9j6hc"]
Sep 11 12:47:13.355348 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.355327    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.357442 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.357418    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.357837 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.357813    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 12:47:13.357837 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.357829    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 12:47:13.357998 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.357886    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 12:47:13.358125 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.358101    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lk9c9\""
Sep 11 12:47:13.359459 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.359436    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 12:47:13.360993 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.359790    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 12:47:13.360993 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.359841    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:13.360993 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.359863    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mhld2\""
Sep 11 12:47:13.360993 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.359961    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:13.360993 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.360107    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 12:47:13.362084 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.362067    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.364261 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.364239    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 12:47:13.364504 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.364484    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.364585 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.364565    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9pcqx\""
Sep 11 12:47:13.364630 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.364579    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 12:47:13.364699 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.364566    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 12:47:13.366752 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.366734    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 12:47:13.367200 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.367180    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 12:47:13.367200 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.367191    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jkqzh\""
Sep 11 12:47:13.367342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.367220    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 12:47:13.367342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.367239    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.367448 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.367387    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 12:47:13.367495 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.367453    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 12:47:13.367549 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.367514    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 12:47:13.367599 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.367569    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 12:47:13.369394 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.369377    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cxqnl\""
Sep 11 12:47:13.369570 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.369555    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 12:47:13.369660 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.369567    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 12:47:13.369864 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.369849    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.371778 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.371745    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 12:47:13.372058 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.372042    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.372120 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.372054    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 12:47:13.372120 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.372090    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ckzwv\""
Sep 11 12:47:13.374294 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374275    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-kubernetes\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.374294 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374289    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 12:47:13.374444 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374301    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-sysctl-conf\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.374444 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374341    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-etc-selinux\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.374444 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374357    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-run-ovn-kubernetes\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.374444 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374410    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-cni-bin\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.374633 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374476    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 12:47:13.374633 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374482    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 12:47:13.374633 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374495    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s9vt6\""
Sep 11 12:47:13.374633 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374502    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 12:47:13.374633 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374448    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8ad4f65c-09c8-4c72-969f-4ac3834c855d-ovnkube-config\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.374633 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374591    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8ad4f65c-09c8-4c72-969f-4ac3834c855d-ovnkube-script-lib\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.374921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374696    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-sysconfig\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.374921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-sys\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.374921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374747    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.374921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-host\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.374921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374815    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b4rn\" (UniqueName: \"kubernetes.io/projected/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-kube-api-access-2b4rn\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.374921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374831    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d9fee76-abbc-4b67-bfaf-3fe94c1e6353-host\") pod \"node-ca-zbccg\" (UID: \"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353\") " pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.374921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-lib-modules\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.374921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374882    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 12:47:13.374921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ad4f65c-09c8-4c72-969f-4ac3834c855d-ovn-node-metrics-cert\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.374921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-systemd\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.374981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-var-lib-kubelet\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375004    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43b71a14-9e3f-425e-b3af-8505175d5f5f-host-slash\") pod \"iptables-alerter-rkngq\" (UID: \"43b71a14-9e3f-425e-b3af-8505175d5f5f\") " pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375026    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-kubelet\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375052    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-run-netns\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375097    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-etc-openvswitch\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375135    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-node-log\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-socket-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375185    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-registration-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-device-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-var-lib-openvswitch\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375265    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8ad4f65c-09c8-4c72-969f-4ac3834c855d-env-overrides\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dj2q\" (UniqueName: \"kubernetes.io/projected/8ad4f65c-09c8-4c72-969f-4ac3834c855d-kube-api-access-5dj2q\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375326    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zww4\" (UniqueName: \"kubernetes.io/projected/439fc464-12e7-415c-922c-c2ed1971e596-kube-api-access-8zww4\") pod \"node-resolver-7lzcr\" (UID: \"439fc464-12e7-415c-922c-c2ed1971e596\") " pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.375357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375351    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twmvc\" (UniqueName: \"kubernetes.io/projected/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-kube-api-access-twmvc\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375376    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-sysctl-d\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375397    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-run\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375413    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f33d987a-f19e-42f2-822b-d752f31a1233-tmp\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375431    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43b71a14-9e3f-425e-b3af-8505175d5f5f-iptables-alerter-script\") pod \"iptables-alerter-rkngq\" (UID: \"43b71a14-9e3f-425e-b3af-8505175d5f5f\") " pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375453    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-systemd-units\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375489    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/439fc464-12e7-415c-922c-c2ed1971e596-tmp-dir\") pod \"node-resolver-7lzcr\" (UID: \"439fc464-12e7-415c-922c-c2ed1971e596\") " pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d9fee76-abbc-4b67-bfaf-3fe94c1e6353-serviceca\") pod \"node-ca-zbccg\" (UID: \"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353\") " pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375552    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f33d987a-f19e-42f2-822b-d752f31a1233-etc-tuned\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375575    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88pl6\" (UniqueName: \"kubernetes.io/projected/43b71a14-9e3f-425e-b3af-8505175d5f5f-kube-api-access-88pl6\") pod \"iptables-alerter-rkngq\" (UID: \"43b71a14-9e3f-425e-b3af-8505175d5f5f\") " pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375597    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-slash\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375616    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-run-systemd\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375674    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-log-socket\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/439fc464-12e7-415c-922c-c2ed1971e596-hosts-file\") pod \"node-resolver-7lzcr\" (UID: \"439fc464-12e7-415c-922c-c2ed1971e596\") " pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375717    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6vjf\" (UniqueName: \"kubernetes.io/projected/5d9fee76-abbc-4b67-bfaf-3fe94c1e6353-kube-api-access-q6vjf\") pod \"node-ca-zbccg\" (UID: \"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353\") " pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wxn5\" (UniqueName: \"kubernetes.io/projected/f33d987a-f19e-42f2-822b-d752f31a1233-kube-api-access-7wxn5\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375769    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.375917 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-sys-fs\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.376450 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375816    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-run-openvswitch\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.376450 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-run-ovn\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.376450 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375889    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-cni-netd\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.376450 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375916    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.376450 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:13.376450 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.375983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-modprobe-d\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.376940 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.376925    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 12:47:13.377074 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.377049    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cwfkk\""
Sep 11 12:47:13.377240 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.377215    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:13.377311 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.377292    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:13.379538 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.379521    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:13.379611 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.379589    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:13.382176 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.382158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:13.382273 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.382243    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 12:47:13.384493 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.384474    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 12:47:13.384599 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.384474    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bnkbp\""
Sep 11 12:47:13.384753 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.384738    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 12:47:13.408260 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.408231    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-phgrm"
Sep 11 12:47:13.415911 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.415886    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-phgrm"
Sep 11 12:47:13.469496 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.469473    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 12:47:13.476353 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476326    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-hostroot\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.476353 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/24a198cf-c041-46dc-b307-f255b181f3ae-multus-daemon-config\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.476549 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476374    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-var-lib-openvswitch\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.476549 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476390    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2a220c7-9039-4263-b46c-0468f32f4d74-system-cni-dir\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.476549 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476436    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-cnibin\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.476549 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-var-lib-openvswitch\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.476549 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476505    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-var-lib-kubelet\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476559    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f33d987a-f19e-42f2-822b-d752f31a1233-tmp\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476587    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43b71a14-9e3f-425e-b3af-8505175d5f5f-iptables-alerter-script\") pod \"iptables-alerter-rkngq\" (UID: \"43b71a14-9e3f-425e-b3af-8505175d5f5f\") " pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/439fc464-12e7-415c-922c-c2ed1971e596-tmp-dir\") pod \"node-resolver-7lzcr\" (UID: \"439fc464-12e7-415c-922c-c2ed1971e596\") " pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476652    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2a220c7-9039-4263-b46c-0468f32f4d74-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476682    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f33d987a-f19e-42f2-822b-d752f31a1233-etc-tuned\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476709    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88pl6\" (UniqueName: \"kubernetes.io/projected/43b71a14-9e3f-425e-b3af-8505175d5f5f-kube-api-access-88pl6\") pod \"iptables-alerter-rkngq\" (UID: \"43b71a14-9e3f-425e-b3af-8505175d5f5f\") " pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-slash\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-log-socket\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476789    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/439fc464-12e7-415c-922c-c2ed1971e596-hosts-file\") pod \"node-resolver-7lzcr\" (UID: \"439fc464-12e7-415c-922c-c2ed1971e596\") " pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476816    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng76k\" (UniqueName: \"kubernetes.io/projected/d2a220c7-9039-4263-b46c-0468f32f4d74-kube-api-access-ng76k\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.476858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-sys-fs\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476868    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-cni-netd\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476879    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-slash\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476935    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/498c45c8-775f-460c-a2f6-18d1e5dfe451-konnectivity-ca\") pod \"konnectivity-agent-9j6hc\" (UID: \"498c45c8-775f-460c-a2f6-18d1e5dfe451\") " pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476944    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-log-socket\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476950    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/439fc464-12e7-415c-922c-c2ed1971e596-tmp-dir\") pod \"node-resolver-7lzcr\" (UID: \"439fc464-12e7-415c-922c-c2ed1971e596\") " pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.476967    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-multus-cni-dir\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477013    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-run-k8s-cni-cncf-io\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477036    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-sys-fs\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477039    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9gwd\" (UniqueName: \"kubernetes.io/projected/24a198cf-c041-46dc-b307-f255b181f3ae-kube-api-access-f9gwd\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-etc-selinux\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477115    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/439fc464-12e7-415c-922c-c2ed1971e596-hosts-file\") pod \"node-resolver-7lzcr\" (UID: \"439fc464-12e7-415c-922c-c2ed1971e596\") " pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477147    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-run-ovn-kubernetes\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477180    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-cni-bin\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.477378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-cni-netd\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8ad4f65c-09c8-4c72-969f-4ac3834c855d-ovnkube-config\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-etc-selinux\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477248    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8ad4f65c-09c8-4c72-969f-4ac3834c855d-ovnkube-script-lib\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477250    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-cni-bin\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477254    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43b71a14-9e3f-425e-b3af-8505175d5f5f-iptables-alerter-script\") pod \"iptables-alerter-rkngq\" (UID: \"43b71a14-9e3f-425e-b3af-8505175d5f5f\") " pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477307    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-run-ovn-kubernetes\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477300    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsgxb\" (UniqueName: \"kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb\") pod \"network-check-target-pdqx9\" (UID: \"5015a5c1-bdb4-47a0-9ad8-325512861d85\") " pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-system-cni-dir\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-sys\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-host\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477406    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-sys\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477463    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-host\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477450    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d9fee76-abbc-4b67-bfaf-3fe94c1e6353-host\") pod \"node-ca-zbccg\" (UID: \"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353\") " pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477503    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d2a220c7-9039-4263-b46c-0468f32f4d74-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477507    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d9fee76-abbc-4b67-bfaf-3fe94c1e6353-host\") pod \"node-ca-zbccg\" (UID: \"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353\") " pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477538    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ad4f65c-09c8-4c72-969f-4ac3834c855d-ovn-node-metrics-cert\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.478144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477600    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477865    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2a220c7-9039-4263-b46c-0468f32f4d74-cnibin\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477890    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-multus-socket-dir-parent\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477901    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8ad4f65c-09c8-4c72-969f-4ac3834c855d-ovnkube-script-lib\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-systemd\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477931    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8ad4f65c-09c8-4c72-969f-4ac3834c855d-ovnkube-config\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43b71a14-9e3f-425e-b3af-8505175d5f5f-host-slash\") pod \"iptables-alerter-rkngq\" (UID: \"43b71a14-9e3f-425e-b3af-8505175d5f5f\") " pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.477967    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-kubelet\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478000    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-kubelet\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478042    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-systemd\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478086    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43b71a14-9e3f-425e-b3af-8505175d5f5f-host-slash\") pod \"iptables-alerter-rkngq\" (UID: \"43b71a14-9e3f-425e-b3af-8505175d5f5f\") " pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-run-netns\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478133    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-etc-openvswitch\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478159    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-host-run-netns\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478175    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-node-log\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478207    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zww4\" (UniqueName: \"kubernetes.io/projected/439fc464-12e7-415c-922c-c2ed1971e596-kube-api-access-8zww4\") pod \"node-resolver-7lzcr\" (UID: \"439fc464-12e7-415c-922c-c2ed1971e596\") " pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478232    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twmvc\" (UniqueName: \"kubernetes.io/projected/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-kube-api-access-twmvc\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:13.479046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478259    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-socket-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-registration-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-device-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478366    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-var-lib-cni-bin\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-var-lib-cni-multus\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-sysctl-d\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-run\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478486    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-systemd-units\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478512    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-etc-openvswitch\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d9fee76-abbc-4b67-bfaf-3fe94c1e6353-serviceca\") pod \"node-ca-zbccg\" (UID: \"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353\") " pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6vjf\" (UniqueName: \"kubernetes.io/projected/5d9fee76-abbc-4b67-bfaf-3fe94c1e6353-kube-api-access-q6vjf\") pod \"node-ca-zbccg\" (UID: \"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353\") " pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478593    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/498c45c8-775f-460c-a2f6-18d1e5dfe451-agent-certs\") pod \"konnectivity-agent-9j6hc\" (UID: \"498c45c8-775f-460c-a2f6-18d1e5dfe451\") " pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478630    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-run-netns\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478670    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-run-systemd\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478697    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478707    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-device-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-etc-kubernetes\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.479827 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wxn5\" (UniqueName: \"kubernetes.io/projected/f33d987a-f19e-42f2-822b-d752f31a1233-kube-api-access-7wxn5\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478776    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-registration-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478874    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-sysctl-d\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d9fee76-abbc-4b67-bfaf-3fe94c1e6353-serviceca\") pod \"node-ca-zbccg\" (UID: \"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353\") " pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.478976    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-node-log\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479046    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-run\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479097    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-systemd-units\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479139    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-run-systemd\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479168    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-socket-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.479190    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.479280    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs podName:7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:13.97924465 +0000 UTC m=+2.041551602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs") pod "network-metrics-daemon-4f7x7" (UID: "7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-run-openvswitch\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479475    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479488    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-run-ovn\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-run-openvswitch\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479523    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-run-multus-certs\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.480591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479530    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8ad4f65c-09c8-4c72-969f-4ac3834c855d-run-ovn\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479556    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-modprobe-d\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479605    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-kubernetes\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479625    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-modprobe-d\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479630    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-sysctl-conf\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479676    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2a220c7-9039-4263-b46c-0468f32f4d74-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479688    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-kubernetes\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479702    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-multus-conf-dir\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479723    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-sysconfig\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479745    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b4rn\" (UniqueName: \"kubernetes.io/projected/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-kube-api-access-2b4rn\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479767    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b32090af-80e4-4ac6-b9b2-da62336a920f-kubelet-config\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479787    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b32090af-80e4-4ac6-b9b2-da62336a920f-dbus\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479809    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-sysconfig\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479810    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24a198cf-c041-46dc-b307-f255b181f3ae-cni-binary-copy\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-lib-modules\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479897    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-var-lib-kubelet\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8ad4f65c-09c8-4c72-969f-4ac3834c855d-env-overrides\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.481342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dj2q\" (UniqueName: \"kubernetes.io/projected/8ad4f65c-09c8-4c72-969f-4ac3834c855d-kube-api-access-5dj2q\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.481876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.479991    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d2a220c7-9039-4263-b46c-0468f32f4d74-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.481876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.480102    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-var-lib-kubelet\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.480178    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-lib-modules\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.480251    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f33d987a-f19e-42f2-822b-d752f31a1233-etc-sysctl-conf\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.480614    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8ad4f65c-09c8-4c72-969f-4ac3834c855d-env-overrides\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.481876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.480893    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f33d987a-f19e-42f2-822b-d752f31a1233-etc-tuned\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.481876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.481084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ad4f65c-09c8-4c72-969f-4ac3834c855d-ovn-node-metrics-cert\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.481876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.481094    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f33d987a-f19e-42f2-822b-d752f31a1233-tmp\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.484873 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.484842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88pl6\" (UniqueName: \"kubernetes.io/projected/43b71a14-9e3f-425e-b3af-8505175d5f5f-kube-api-access-88pl6\") pod \"iptables-alerter-rkngq\" (UID: \"43b71a14-9e3f-425e-b3af-8505175d5f5f\") " pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.486278 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.486259    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twmvc\" (UniqueName: \"kubernetes.io/projected/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-kube-api-access-twmvc\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:13.489773 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.489738    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zww4\" (UniqueName: \"kubernetes.io/projected/439fc464-12e7-415c-922c-c2ed1971e596-kube-api-access-8zww4\") pod \"node-resolver-7lzcr\" (UID: \"439fc464-12e7-415c-922c-c2ed1971e596\") " pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.490898 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.490872    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b4rn\" (UniqueName: \"kubernetes.io/projected/1a1aaddc-3781-4802-aca3-d9ba0fd9e83f-kube-api-access-2b4rn\") pod \"aws-ebs-csi-driver-node-gj2lh\" (UID: \"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.491184 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.491160    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dj2q\" (UniqueName: \"kubernetes.io/projected/8ad4f65c-09c8-4c72-969f-4ac3834c855d-kube-api-access-5dj2q\") pod \"ovnkube-node-dscjx\" (UID: \"8ad4f65c-09c8-4c72-969f-4ac3834c855d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.491448 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.491423    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wxn5\" (UniqueName: \"kubernetes.io/projected/f33d987a-f19e-42f2-822b-d752f31a1233-kube-api-access-7wxn5\") pod \"tuned-rx64q\" (UID: \"f33d987a-f19e-42f2-822b-d752f31a1233\") " pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.491545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.491529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6vjf\" (UniqueName: \"kubernetes.io/projected/5d9fee76-abbc-4b67-bfaf-3fe94c1e6353-kube-api-access-q6vjf\") pod \"node-ca-zbccg\" (UID: \"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353\") " pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.580448 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580421    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-var-lib-cni-bin\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580562 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580453    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-var-lib-cni-multus\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580562 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580477    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/498c45c8-775f-460c-a2f6-18d1e5dfe451-agent-certs\") pod \"konnectivity-agent-9j6hc\" (UID: \"498c45c8-775f-460c-a2f6-18d1e5dfe451\") " pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:13.580562 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580500    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-run-netns\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580562 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580519    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-var-lib-cni-bin\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580562 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-run-netns\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580562 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-var-lib-cni-multus\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-etc-kubernetes\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580603    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-run-multus-certs\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580624    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2a220c7-9039-4263-b46c-0468f32f4d74-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580636    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-etc-kubernetes\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580662    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-run-multus-certs\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-multus-conf-dir\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580712    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b32090af-80e4-4ac6-b9b2-da62336a920f-kubelet-config\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580678    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-multus-conf-dir\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b32090af-80e4-4ac6-b9b2-da62336a920f-dbus\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24a198cf-c041-46dc-b307-f255b181f3ae-cni-binary-copy\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b32090af-80e4-4ac6-b9b2-da62336a920f-kubelet-config\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d2a220c7-9039-4263-b46c-0468f32f4d74-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-hostroot\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580823    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/24a198cf-c041-46dc-b307-f255b181f3ae-multus-daemon-config\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2a220c7-9039-4263-b46c-0468f32f4d74-system-cni-dir\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-cnibin\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.580876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580871    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-var-lib-kubelet\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580897    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2a220c7-9039-4263-b46c-0468f32f4d74-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580949    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-var-lib-kubelet\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-hostroot\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.580979    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-cnibin\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng76k\" (UniqueName: \"kubernetes.io/projected/d2a220c7-9039-4263-b46c-0468f32f4d74-kube-api-access-ng76k\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581055    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/498c45c8-775f-460c-a2f6-18d1e5dfe451-konnectivity-ca\") pod \"konnectivity-agent-9j6hc\" (UID: \"498c45c8-775f-460c-a2f6-18d1e5dfe451\") " pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-multus-cni-dir\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2a220c7-9039-4263-b46c-0468f32f4d74-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581108    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-run-k8s-cni-cncf-io\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581132    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9gwd\" (UniqueName: \"kubernetes.io/projected/24a198cf-c041-46dc-b307-f255b181f3ae-kube-api-access-f9gwd\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgxb\" (UniqueName: \"kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb\") pod \"network-check-target-pdqx9\" (UID: \"5015a5c1-bdb4-47a0-9ad8-325512861d85\") " pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581192    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2a220c7-9039-4263-b46c-0468f32f4d74-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-system-cni-dir\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d2a220c7-9039-4263-b46c-0468f32f4d74-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581240    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b32090af-80e4-4ac6-b9b2-da62336a920f-dbus\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2a220c7-9039-4263-b46c-0468f32f4d74-system-cni-dir\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.581728 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581262    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581285    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d2a220c7-9039-4263-b46c-0468f32f4d74-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581306    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2a220c7-9039-4263-b46c-0468f32f4d74-cnibin\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.581339    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-multus-socket-dir-parent\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581373    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/24a198cf-c041-46dc-b307-f255b181f3ae-multus-daemon-config\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.581403    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret podName:b32090af-80e4-4ac6-b9b2-da62336a920f nodeName:}" failed. No retries permitted until 2026-09-11 12:47:14.08138583 +0000 UTC m=+2.143692780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret") pod "global-pull-secret-syncer-2btsm" (UID: "b32090af-80e4-4ac6-b9b2-da62336a920f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581539    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-host-run-k8s-cni-cncf-io\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581557    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-multus-cni-dir\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581615    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-system-cni-dir\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/498c45c8-775f-460c-a2f6-18d1e5dfe451-konnectivity-ca\") pod \"konnectivity-agent-9j6hc\" (UID: \"498c45c8-775f-460c-a2f6-18d1e5dfe451\") " pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/24a198cf-c041-46dc-b307-f255b181f3ae-multus-socket-dir-parent\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581751    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2a220c7-9039-4263-b46c-0468f32f4d74-cnibin\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.581844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24a198cf-c041-46dc-b307-f255b181f3ae-cni-binary-copy\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.582396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.582378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d2a220c7-9039-4263-b46c-0468f32f4d74-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.583673 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.583653    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/498c45c8-775f-460c-a2f6-18d1e5dfe451-agent-certs\") pod \"konnectivity-agent-9j6hc\" (UID: \"498c45c8-775f-460c-a2f6-18d1e5dfe451\") " pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:13.588669 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.588634    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:47:13.588669 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.588671    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:47:13.588867 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.588681    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qsgxb for pod openshift-network-diagnostics/network-check-target-pdqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:13.588867 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.588725    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb podName:5015a5c1-bdb4-47a0-9ad8-325512861d85 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:14.088712767 +0000 UTC m=+2.151019703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qsgxb" (UniqueName: "kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb") pod "network-check-target-pdqx9" (UID: "5015a5c1-bdb4-47a0-9ad8-325512861d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:13.588987 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.588911    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c21e6ec17475096785af4f25e5a0af6.slice/crio-9f847f1ee99fea92e9f351945a05e59d38940402b8491bb1b587f550218c2df9 WatchSource:0}: Error finding container 9f847f1ee99fea92e9f351945a05e59d38940402b8491bb1b587f550218c2df9: Status 404 returned error can't find the container with id 9f847f1ee99fea92e9f351945a05e59d38940402b8491bb1b587f550218c2df9
Sep 11 12:47:13.589665 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.589625    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0dbe8941313665e9751104dfb30bc72.slice/crio-dbe2631947f8d328e2fe8c35aa7a1ea0cd419d6f569ab6f691d458c42eda6d6f WatchSource:0}: Error finding container dbe2631947f8d328e2fe8c35aa7a1ea0cd419d6f569ab6f691d458c42eda6d6f: Status 404 returned error can't find the container with id dbe2631947f8d328e2fe8c35aa7a1ea0cd419d6f569ab6f691d458c42eda6d6f
Sep 11 12:47:13.590505 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.590482    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9gwd\" (UniqueName: \"kubernetes.io/projected/24a198cf-c041-46dc-b307-f255b181f3ae-kube-api-access-f9gwd\") pod \"multus-tv9wd\" (UID: \"24a198cf-c041-46dc-b307-f255b181f3ae\") " pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.590705 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.590690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng76k\" (UniqueName: \"kubernetes.io/projected/d2a220c7-9039-4263-b46c-0468f32f4d74-kube-api-access-ng76k\") pod \"multus-additional-cni-plugins-jsq92\" (UID: \"d2a220c7-9039-4263-b46c-0468f32f4d74\") " pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.594507 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.594487    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:47:13.686147 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.686111    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh"
Sep 11 12:47:13.695525 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.695496    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a1aaddc_3781_4802_aca3_d9ba0fd9e83f.slice/crio-c80bd9ec0c9eb8d349352c69d004f7c12004c0be232672ab33c68d83dbf446b1 WatchSource:0}: Error finding container c80bd9ec0c9eb8d349352c69d004f7c12004c0be232672ab33c68d83dbf446b1: Status 404 returned error can't find the container with id c80bd9ec0c9eb8d349352c69d004f7c12004c0be232672ab33c68d83dbf446b1
Sep 11 12:47:13.701357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.701336    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zbccg"
Sep 11 12:47:13.709602 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.709578    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d9fee76_abbc_4b67_bfaf_3fe94c1e6353.slice/crio-60201b904e817727d57db6052ae58635961903aacc18f2e69ed7bce513e9995e WatchSource:0}: Error finding container 60201b904e817727d57db6052ae58635961903aacc18f2e69ed7bce513e9995e: Status 404 returned error can't find the container with id 60201b904e817727d57db6052ae58635961903aacc18f2e69ed7bce513e9995e
Sep 11 12:47:13.725109 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.725081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rkngq"
Sep 11 12:47:13.729151 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.729120    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:13.734515 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.734490    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b71a14_9e3f_425e_b3af_8505175d5f5f.slice/crio-6180d863af8a92b6e3ad811f0921467d40eb9b954dbe90d09a3135412f68cf63 WatchSource:0}: Error finding container 6180d863af8a92b6e3ad811f0921467d40eb9b954dbe90d09a3135412f68cf63: Status 404 returned error can't find the container with id 6180d863af8a92b6e3ad811f0921467d40eb9b954dbe90d09a3135412f68cf63
Sep 11 12:47:13.738823 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.738801    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad4f65c_09c8_4c72_969f_4ac3834c855d.slice/crio-106af2761248f01bef82d9f0f1c8cd1ed7371af791769d7496768efa44508eb2 WatchSource:0}: Error finding container 106af2761248f01bef82d9f0f1c8cd1ed7371af791769d7496768efa44508eb2: Status 404 returned error can't find the container with id 106af2761248f01bef82d9f0f1c8cd1ed7371af791769d7496768efa44508eb2
Sep 11 12:47:13.752461 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.752440    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rx64q"
Sep 11 12:47:13.761130 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.761104    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf33d987a_f19e_42f2_822b_d752f31a1233.slice/crio-25b90b03776b1a27f0ac9c4b50d8e90cb6421fc42182614a174258fb2889212d WatchSource:0}: Error finding container 25b90b03776b1a27f0ac9c4b50d8e90cb6421fc42182614a174258fb2889212d: Status 404 returned error can't find the container with id 25b90b03776b1a27f0ac9c4b50d8e90cb6421fc42182614a174258fb2889212d
Sep 11 12:47:13.766304 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.766286    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7lzcr"
Sep 11 12:47:13.774458 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.774438    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod439fc464_12e7_415c_922c_c2ed1971e596.slice/crio-b3b84bd15a6fd92572a2e937661abf8f538326b44a95717c99c6e50a78d8b60c WatchSource:0}: Error finding container b3b84bd15a6fd92572a2e937661abf8f538326b44a95717c99c6e50a78d8b60c: Status 404 returned error can't find the container with id b3b84bd15a6fd92572a2e937661abf8f538326b44a95717c99c6e50a78d8b60c
Sep 11 12:47:13.781105 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.781083    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jsq92"
Sep 11 12:47:13.789757 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.789730    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a220c7_9039_4263_b46c_0468f32f4d74.slice/crio-b8d85b48442387820adbee60882c3b7fc51c9e8663ff915827230cfab24b4ad6 WatchSource:0}: Error finding container b8d85b48442387820adbee60882c3b7fc51c9e8663ff915827230cfab24b4ad6: Status 404 returned error can't find the container with id b8d85b48442387820adbee60882c3b7fc51c9e8663ff915827230cfab24b4ad6
Sep 11 12:47:13.803357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.803330    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tv9wd"
Sep 11 12:47:13.806986 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.806962    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:13.813159 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.813133    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a198cf_c041_46dc_b307_f255b181f3ae.slice/crio-62cbb5628c7fd496c7e0809fed50cf1519c14a5dd31291211be170895b87ed13 WatchSource:0}: Error finding container 62cbb5628c7fd496c7e0809fed50cf1519c14a5dd31291211be170895b87ed13: Status 404 returned error can't find the container with id 62cbb5628c7fd496c7e0809fed50cf1519c14a5dd31291211be170895b87ed13
Sep 11 12:47:13.816121 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:13.816092    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod498c45c8_775f_460c_a2f6_18d1e5dfe451.slice/crio-9dd3fa8fcbb4144c4b3e4a852fcc23ad39697fce764a587111b600eb562e90bb WatchSource:0}: Error finding container 9dd3fa8fcbb4144c4b3e4a852fcc23ad39697fce764a587111b600eb562e90bb: Status 404 returned error can't find the container with id 9dd3fa8fcbb4144c4b3e4a852fcc23ad39697fce764a587111b600eb562e90bb
Sep 11 12:47:13.983842 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:13.983763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:13.984000 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.983872    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:13.984000 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:13.983928    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs podName:7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:14.983911694 +0000 UTC m=+3.046218648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs") pod "network-metrics-daemon-4f7x7" (UID: "7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:14.084412 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.084311    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:14.084412 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:14.084406    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:14.084669 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:14.084474    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret podName:b32090af-80e4-4ac6-b9b2-da62336a920f nodeName:}" failed. No retries permitted until 2026-09-11 12:47:15.084446014 +0000 UTC m=+3.146752954 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret") pod "global-pull-secret-syncer-2btsm" (UID: "b32090af-80e4-4ac6-b9b2-da62336a920f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:14.129079 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.129038    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:47:14.184747 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.184702    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgxb\" (UniqueName: \"kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb\") pod \"network-check-target-pdqx9\" (UID: \"5015a5c1-bdb4-47a0-9ad8-325512861d85\") " pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:14.185008 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:14.184943    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:47:14.185008 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:14.184966    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:47:14.185008 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:14.184980    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qsgxb for pod openshift-network-diagnostics/network-check-target-pdqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:14.185164 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:14.185053    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb podName:5015a5c1-bdb4-47a0-9ad8-325512861d85 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:15.185033934 +0000 UTC m=+3.247340871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsgxb" (UniqueName: "kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb") pod "network-check-target-pdqx9" (UID: "5015a5c1-bdb4-47a0-9ad8-325512861d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:14.419370 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.419201    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 12:42:14 +0000 UTC" deadline="2028-04-23 20:37:37.835412943 +0000 UTC"
Sep 11 12:47:14.419370 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.419268    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14167h50m23.41614944s"
Sep 11 12:47:14.477921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.477759    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:14.477921 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:14.477899    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:14.506242 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.506181    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9j6hc" event={"ID":"498c45c8-775f-460c-a2f6-18d1e5dfe451","Type":"ContainerStarted","Data":"9dd3fa8fcbb4144c4b3e4a852fcc23ad39697fce764a587111b600eb562e90bb"}
Sep 11 12:47:14.508946 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.508909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tv9wd" event={"ID":"24a198cf-c041-46dc-b307-f255b181f3ae","Type":"ContainerStarted","Data":"62cbb5628c7fd496c7e0809fed50cf1519c14a5dd31291211be170895b87ed13"}
Sep 11 12:47:14.521423 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.521384    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7lzcr" event={"ID":"439fc464-12e7-415c-922c-c2ed1971e596","Type":"ContainerStarted","Data":"b3b84bd15a6fd92572a2e937661abf8f538326b44a95717c99c6e50a78d8b60c"}
Sep 11 12:47:14.536189 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.536160    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:47:14.540948 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.540909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh" event={"ID":"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f","Type":"ContainerStarted","Data":"c80bd9ec0c9eb8d349352c69d004f7c12004c0be232672ab33c68d83dbf446b1"}
Sep 11 12:47:14.549657 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.549604    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal" event={"ID":"f0dbe8941313665e9751104dfb30bc72","Type":"ContainerStarted","Data":"dbe2631947f8d328e2fe8c35aa7a1ea0cd419d6f569ab6f691d458c42eda6d6f"}
Sep 11 12:47:14.559063 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.558874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsq92" event={"ID":"d2a220c7-9039-4263-b46c-0468f32f4d74","Type":"ContainerStarted","Data":"b8d85b48442387820adbee60882c3b7fc51c9e8663ff915827230cfab24b4ad6"}
Sep 11 12:47:14.571869 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.571776    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rx64q" event={"ID":"f33d987a-f19e-42f2-822b-d752f31a1233","Type":"ContainerStarted","Data":"25b90b03776b1a27f0ac9c4b50d8e90cb6421fc42182614a174258fb2889212d"}
Sep 11 12:47:14.577316 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.577134    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:47:14.599659 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.599597    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" event={"ID":"8ad4f65c-09c8-4c72-969f-4ac3834c855d","Type":"ContainerStarted","Data":"106af2761248f01bef82d9f0f1c8cd1ed7371af791769d7496768efa44508eb2"}
Sep 11 12:47:14.627714 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.627630    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rkngq" event={"ID":"43b71a14-9e3f-425e-b3af-8505175d5f5f","Type":"ContainerStarted","Data":"6180d863af8a92b6e3ad811f0921467d40eb9b954dbe90d09a3135412f68cf63"}
Sep 11 12:47:14.654926 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.654839    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zbccg" event={"ID":"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353","Type":"ContainerStarted","Data":"60201b904e817727d57db6052ae58635961903aacc18f2e69ed7bce513e9995e"}
Sep 11 12:47:14.657360 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.657282    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal" event={"ID":"1c21e6ec17475096785af4f25e5a0af6","Type":"ContainerStarted","Data":"9f847f1ee99fea92e9f351945a05e59d38940402b8491bb1b587f550218c2df9"}
Sep 11 12:47:14.991605 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:14.991561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:14.991850 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:14.991776    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:14.991850 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:14.991848    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs podName:7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:16.991825912 +0000 UTC m=+5.054132850 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs") pod "network-metrics-daemon-4f7x7" (UID: "7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:15.092376 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:15.092320    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:15.092578 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:15.092558    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:15.092664 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:15.092635    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret podName:b32090af-80e4-4ac6-b9b2-da62336a920f nodeName:}" failed. No retries permitted until 2026-09-11 12:47:17.092614541 +0000 UTC m=+5.154921500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret") pod "global-pull-secret-syncer-2btsm" (UID: "b32090af-80e4-4ac6-b9b2-da62336a920f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:15.193228 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:15.193164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgxb\" (UniqueName: \"kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb\") pod \"network-check-target-pdqx9\" (UID: \"5015a5c1-bdb4-47a0-9ad8-325512861d85\") " pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:15.193420 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:15.193377    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:47:15.193420 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:15.193396    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:47:15.193420 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:15.193409    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qsgxb for pod openshift-network-diagnostics/network-check-target-pdqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:15.193587 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:15.193471    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb podName:5015a5c1-bdb4-47a0-9ad8-325512861d85 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:17.193451349 +0000 UTC m=+5.255758315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsgxb" (UniqueName: "kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb") pod "network-check-target-pdqx9" (UID: "5015a5c1-bdb4-47a0-9ad8-325512861d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:15.419973 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:15.419862    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 12:42:14 +0000 UTC" deadline="2028-04-12 07:41:50.171872295 +0000 UTC"
Sep 11 12:47:15.419973 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:15.419907    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13890h54m34.751970097s"
Sep 11 12:47:15.477661 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:15.477617    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:15.477858 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:15.477776    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:15.478256 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:15.478225    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:15.478380 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:15.478350    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:16.481043 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:16.480192    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:16.481043 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:16.480347    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:17.007850 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:17.007806    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:17.008027 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:17.007994    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:17.008083 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:17.008069    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs podName:7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:21.008047782 +0000 UTC m=+9.070354745 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs") pod "network-metrics-daemon-4f7x7" (UID: "7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:17.108245 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:17.108201    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:17.108495 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:17.108409    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:17.108495 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:17.108490    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret podName:b32090af-80e4-4ac6-b9b2-da62336a920f nodeName:}" failed. No retries permitted until 2026-09-11 12:47:21.108470389 +0000 UTC m=+9.170777332 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret") pod "global-pull-secret-syncer-2btsm" (UID: "b32090af-80e4-4ac6-b9b2-da62336a920f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:17.209721 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:17.209592    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgxb\" (UniqueName: \"kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb\") pod \"network-check-target-pdqx9\" (UID: \"5015a5c1-bdb4-47a0-9ad8-325512861d85\") " pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:17.209966 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:17.209896    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:47:17.209966 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:17.209923    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:47:17.209966 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:17.209938    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qsgxb for pod openshift-network-diagnostics/network-check-target-pdqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:17.210144 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:17.210003    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb podName:5015a5c1-bdb4-47a0-9ad8-325512861d85 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:21.209984531 +0000 UTC m=+9.272291471 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsgxb" (UniqueName: "kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb") pod "network-check-target-pdqx9" (UID: "5015a5c1-bdb4-47a0-9ad8-325512861d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:17.477224 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:17.477129    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:17.477413 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:17.477293    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:17.477776 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:17.477755    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:17.477932 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:17.477884    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:18.477026 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:18.476983    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:18.477572 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:18.477143    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:19.477352 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:19.477314    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:19.477927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:19.477359    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:19.477927 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:19.477468    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:19.477927 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:19.477615    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:20.478624 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:20.478036    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:20.478624 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:20.478180    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:21.041529 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:21.041086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:21.041529 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:21.041306    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:21.041529 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:21.041374    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs podName:7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:29.041354123 +0000 UTC m=+17.103661080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs") pod "network-metrics-daemon-4f7x7" (UID: "7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:21.142054 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:21.142005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:21.142252 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:21.142203    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:21.142317 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:21.142293    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret podName:b32090af-80e4-4ac6-b9b2-da62336a920f nodeName:}" failed. No retries permitted until 2026-09-11 12:47:29.142268437 +0000 UTC m=+17.204575390 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret") pod "global-pull-secret-syncer-2btsm" (UID: "b32090af-80e4-4ac6-b9b2-da62336a920f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:21.243176 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:21.243134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgxb\" (UniqueName: \"kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb\") pod \"network-check-target-pdqx9\" (UID: \"5015a5c1-bdb4-47a0-9ad8-325512861d85\") " pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:21.243389 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:21.243355    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:47:21.243389 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:21.243389    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:47:21.243526 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:21.243405    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qsgxb for pod openshift-network-diagnostics/network-check-target-pdqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:21.243526 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:21.243471    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb podName:5015a5c1-bdb4-47a0-9ad8-325512861d85 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:29.243449704 +0000 UTC m=+17.305756644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsgxb" (UniqueName: "kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb") pod "network-check-target-pdqx9" (UID: "5015a5c1-bdb4-47a0-9ad8-325512861d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:21.476965 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:21.476876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:21.476965 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:21.476917    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:21.477185 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:21.477004    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:21.477185 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:21.477096    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:22.478644 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:22.478094    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:22.478644 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:22.478228    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:23.477130 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:23.477092    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:23.477130 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:23.477130    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:23.477394 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:23.477258    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:23.477394 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:23.477351    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:24.476914 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:24.476869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:24.477415 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:24.477025    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:25.477584 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:25.477550    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:25.477584 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:25.477584    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:25.478060 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:25.477723    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:25.478060 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:25.477872    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:26.477275 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:26.477202    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:26.477428 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:26.477312    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:27.477090 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:27.477055    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:27.477573 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:27.477097    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:27.477573 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:27.477171    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:27.477573 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:27.477303    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:28.477527 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:28.477491    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:28.478012 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:28.477618    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:29.105949 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:29.105909    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:29.106200 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:29.106093    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:29.106200 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:29.106158    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs podName:7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:45.106138377 +0000 UTC m=+33.168445315 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs") pod "network-metrics-daemon-4f7x7" (UID: "7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:29.207235 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:29.207194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:29.207431 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:29.207352    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:29.207431 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:29.207421    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret podName:b32090af-80e4-4ac6-b9b2-da62336a920f nodeName:}" failed. No retries permitted until 2026-09-11 12:47:45.207401928 +0000 UTC m=+33.269708864 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret") pod "global-pull-secret-syncer-2btsm" (UID: "b32090af-80e4-4ac6-b9b2-da62336a920f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:29.307558 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:29.307515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgxb\" (UniqueName: \"kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb\") pod \"network-check-target-pdqx9\" (UID: \"5015a5c1-bdb4-47a0-9ad8-325512861d85\") " pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:29.307748 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:29.307682    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:47:29.307748 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:29.307697    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:47:29.307748 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:29.307707    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qsgxb for pod openshift-network-diagnostics/network-check-target-pdqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:29.307852 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:29.307759    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb podName:5015a5c1-bdb4-47a0-9ad8-325512861d85 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:45.307746449 +0000 UTC m=+33.370053385 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsgxb" (UniqueName: "kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb") pod "network-check-target-pdqx9" (UID: "5015a5c1-bdb4-47a0-9ad8-325512861d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:29.477248 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:29.477163    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:29.477445 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:29.477275    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:29.477445 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:29.477332    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:29.477445 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:29.477432    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:30.477707 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:30.477661    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:30.478181 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:30.477809    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:31.477594 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:31.477551    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:31.477594 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:31.477582    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:31.478083 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:31.477736    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:31.478083 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:31.477890    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:32.477604 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:32.477573    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:32.477846 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:32.477705    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:33.476849 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:33.476816    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:33.476999 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:33.476926    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:33.476999 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:33.476950    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:33.477086 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:33.477047    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:34.479809 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.478841    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:34.479809 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:34.479395    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:34.707531 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.707496    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh" event={"ID":"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f","Type":"ContainerStarted","Data":"5686a1c8b706a9f9f652d94c5a1ac4b51ba44fc43e5c42722fd9155a30d0d3c9"}
Sep 11 12:47:34.708991 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.708968    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal" event={"ID":"f0dbe8941313665e9751104dfb30bc72","Type":"ContainerStarted","Data":"bf5d0dfe6d9de22b68dec71fae4b6ee1f0cf078f09aaf1041fc83f6c68352606"}
Sep 11 12:47:34.710453 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.710431    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2a220c7-9039-4263-b46c-0468f32f4d74" containerID="c79d90d53278a6148d566100b825b7f109f733a89e49b514e5b4d365ec0a3f57" exitCode=0
Sep 11 12:47:34.710541 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.710494    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsq92" event={"ID":"d2a220c7-9039-4263-b46c-0468f32f4d74","Type":"ContainerDied","Data":"c79d90d53278a6148d566100b825b7f109f733a89e49b514e5b4d365ec0a3f57"}
Sep 11 12:47:34.711909 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.711890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rx64q" event={"ID":"f33d987a-f19e-42f2-822b-d752f31a1233","Type":"ContainerStarted","Data":"fa24e93b38ea5f0636d6ab8eadfe640a8b8142f350c0a151e64cd91d2a9714cb"}
Sep 11 12:47:34.714529 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.714512    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 12:47:34.714861 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.714838    2564 generic.go:356] "Generic (PLEG): container finished" podID="8ad4f65c-09c8-4c72-969f-4ac3834c855d" containerID="be0d16b676918de951b43c4ea30e8d8633109054032b33d759adc60fe3b78112" exitCode=1
Sep 11 12:47:34.714954 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.714902    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" event={"ID":"8ad4f65c-09c8-4c72-969f-4ac3834c855d","Type":"ContainerStarted","Data":"d517142e23d31569c33133d78c8ddd956698cf4c4e577438031a88e6491c8dfc"}
Sep 11 12:47:34.715005 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.714960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" event={"ID":"8ad4f65c-09c8-4c72-969f-4ac3834c855d","Type":"ContainerStarted","Data":"e042c006d59087f1419bd8bf2ca0c7b30f154d9e4158ca0303f15e4f4ce66a32"}
Sep 11 12:47:34.715005 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.714993    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" event={"ID":"8ad4f65c-09c8-4c72-969f-4ac3834c855d","Type":"ContainerStarted","Data":"1cef3fcfd075801c191efb988d58f7e66103e86ab96be36a169f7c78cd4e359b"}
Sep 11 12:47:34.715078 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.715032    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" event={"ID":"8ad4f65c-09c8-4c72-969f-4ac3834c855d","Type":"ContainerStarted","Data":"0cf1e6289ffdb55e8c1f80db02b6417ac97f34f5da1ae2a8a82adf61c9f153f3"}
Sep 11 12:47:34.715078 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.715069    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" event={"ID":"8ad4f65c-09c8-4c72-969f-4ac3834c855d","Type":"ContainerDied","Data":"be0d16b676918de951b43c4ea30e8d8633109054032b33d759adc60fe3b78112"}
Sep 11 12:47:34.715167 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.715100    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" event={"ID":"8ad4f65c-09c8-4c72-969f-4ac3834c855d","Type":"ContainerStarted","Data":"ec22d3bfdc7c348afb990dd666fc085548762b4a8a805d81bf04024b5000a6e4"}
Sep 11 12:47:34.716392 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.716376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zbccg" event={"ID":"5d9fee76-abbc-4b67-bfaf-3fe94c1e6353","Type":"ContainerStarted","Data":"be70fa23a0e73498e51addcb934637a9ab761583cbffc31d3a7089508e2656eb"}
Sep 11 12:47:34.717985 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.717966    2564 generic.go:356] "Generic (PLEG): container finished" podID="1c21e6ec17475096785af4f25e5a0af6" containerID="bebae9ef7f8dd43a391bc705fddf77dafcdc52a6aa41ffe2ab279819f97b6788" exitCode=0
Sep 11 12:47:34.718082 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.718029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal" event={"ID":"1c21e6ec17475096785af4f25e5a0af6","Type":"ContainerDied","Data":"bebae9ef7f8dd43a391bc705fddf77dafcdc52a6aa41ffe2ab279819f97b6788"}
Sep 11 12:47:34.719531 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.719506    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9j6hc" event={"ID":"498c45c8-775f-460c-a2f6-18d1e5dfe451","Type":"ContainerStarted","Data":"6996aa4f98812189a51f8076bbe363c8005a73bd03d4b61e39be01e44eb4ab2e"}
Sep 11 12:47:34.721008 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.720988    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tv9wd" event={"ID":"24a198cf-c041-46dc-b307-f255b181f3ae","Type":"ContainerStarted","Data":"930be56e7f1f37550059d9c7fe7f680bd775d87c414a895ad4cf1f899adfc1a4"}
Sep 11 12:47:34.722298 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.722276    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7lzcr" event={"ID":"439fc464-12e7-415c-922c-c2ed1971e596","Type":"ContainerStarted","Data":"120a96003df54ccb96d434cddc7e98bb967ba06ecf686f77e29110dcd10f97ae"}
Sep 11 12:47:34.737737 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.737685    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zbccg" podStartSLOduration=1.917951527 podStartE2EDuration="21.737662984s" totalImagesPullingTime="19.819711457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:34.737630736 +0000 UTC m=+22.799937705" watchObservedRunningTime="2026-09-11 12:47:34.737662984 +0000 UTC m=+22.799969936"
Sep 11 12:47:34.738017 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.737985    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-55.ec2.internal" podStartSLOduration=21.737976771 podStartE2EDuration="21.737976771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:34.722491037 +0000 UTC m=+22.784797996" watchObservedRunningTime="2026-09-11 12:47:34.737976771 +0000 UTC m=+22.800283728"
Sep 11 12:47:34.753167 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.753105    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rx64q" podStartSLOduration=1.9648437520000002 podStartE2EDuration="21.753084348s" totalImagesPullingTime="19.788240596s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:34.753006789 +0000 UTC m=+22.815313748" watchObservedRunningTime="2026-09-11 12:47:34.753084348 +0000 UTC m=+22.815391306"
Sep 11 12:47:34.788753 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.788706    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tv9wd" podStartSLOduration=1.232503863 podStartE2EDuration="21.788689012s" totalImagesPullingTime="20.556185149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:34.788228718 +0000 UTC m=+22.850535677" watchObservedRunningTime="2026-09-11 12:47:34.788689012 +0000 UTC m=+22.850995979"
Sep 11 12:47:34.802681 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.802622    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7lzcr" podStartSLOduration=2.029370442 podStartE2EDuration="21.802605614s" totalImagesPullingTime="19.773235172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:34.802347941 +0000 UTC m=+22.864654883" watchObservedRunningTime="2026-09-11 12:47:34.802605614 +0000 UTC m=+22.864912571"
Sep 11 12:47:34.815864 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:34.815817    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9j6hc" podStartSLOduration=2.163372317 podStartE2EDuration="21.815800898s" totalImagesPullingTime="19.652428581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:34.815762803 +0000 UTC m=+22.878069762" watchObservedRunningTime="2026-09-11 12:47:34.815800898 +0000 UTC m=+22.878107853"
Sep 11 12:47:35.477596 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:35.477176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:35.477911 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:35.477618    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:35.477911 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:35.477200    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:35.477911 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:35.477782    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:35.726859 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:35.726821    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal" event={"ID":"1c21e6ec17475096785af4f25e5a0af6","Type":"ContainerStarted","Data":"c81b3c796314763a2666ed093df646717a85fa743e5f379f0ca4ee96e813e441"}
Sep 11 12:47:35.728627 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:35.728555    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rkngq" event={"ID":"43b71a14-9e3f-425e-b3af-8505175d5f5f","Type":"ContainerStarted","Data":"ece354b977d9080ed0d8f65144c263c5acf04235fa04832c90aa689bef8ca648"}
Sep 11 12:47:35.755994 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:35.755946    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-55.ec2.internal" podStartSLOduration=22.755923412 podStartE2EDuration="22.755923412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:35.741538746 +0000 UTC m=+23.803845706" watchObservedRunningTime="2026-09-11 12:47:35.755923412 +0000 UTC m=+23.818230370"
Sep 11 12:47:36.010915 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:36.010886    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 12:47:36.448945 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:36.448780    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T12:47:36.01090849Z","UUID":"3cf8c7d5-9757-499b-a13f-ee238284eff0","Handler":null,"Name":"","Endpoint":""}
Sep 11 12:47:36.450429 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:36.450411    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 12:47:36.450541 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:36.450438    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 12:47:36.477303 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:36.477262    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:36.477481 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:36.477408    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:36.735507 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:36.735425    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 12:47:36.736043 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:36.735891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" event={"ID":"8ad4f65c-09c8-4c72-969f-4ac3834c855d","Type":"ContainerStarted","Data":"7b6c3bc6c353dc089a0f379466255fd6ec55b5d79bc029ee71249543c7893372"}
Sep 11 12:47:36.738203 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:36.738167    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh" event={"ID":"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f","Type":"ContainerStarted","Data":"725f20bca0bc4654aa7a921f884fb074912aa5223d641f7bd9740f1b3bdc49c5"}
Sep 11 12:47:37.173136 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:37.173055    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:37.173757 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:37.173730    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:37.190104 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:37.190047    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rkngq" podStartSLOduration=4.370810259 podStartE2EDuration="24.190027368s" totalImagesPullingTime="19.819217109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:35.755757392 +0000 UTC m=+23.818064350" watchObservedRunningTime="2026-09-11 12:47:37.190027368 +0000 UTC m=+25.252334330"
Sep 11 12:47:37.476949 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:37.476862    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:37.476949 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:37.476902    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:37.477125 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:37.476987    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:37.477125 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:37.477110    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:37.743419 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:37.743328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh" event={"ID":"1a1aaddc-3781-4802-aca3-d9ba0fd9e83f","Type":"ContainerStarted","Data":"d6eb2a783adb03157baa5f3551663a4cb122dcfa9a7593569b685be6ccb355d5"}
Sep 11 12:47:37.743848 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:37.743568    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:37.744085 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:37.744063    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9j6hc"
Sep 11 12:47:37.772248 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:37.772200    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gj2lh" podStartSLOduration=2.430483453 podStartE2EDuration="24.772184337s" totalImagesPullingTime="22.341700884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:37.771630248 +0000 UTC m=+25.833937205" watchObservedRunningTime="2026-09-11 12:47:37.772184337 +0000 UTC m=+25.834491334"
Sep 11 12:47:38.477636 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:38.477598    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:38.477899 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:38.477746    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:39.477631 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:39.477381    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:39.478105 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:39.477406    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:39.478105 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:39.477717    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:39.478105 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:39.477776    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:39.750247 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:39.750161    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2a220c7-9039-4263-b46c-0468f32f4d74" containerID="b2951ac7fcfa01a10676be91ef4c6b427ccbdc7cdbc74ab6395a70f29b2a42d9" exitCode=0
Sep 11 12:47:39.750247 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:39.750222    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsq92" event={"ID":"d2a220c7-9039-4263-b46c-0468f32f4d74","Type":"ContainerDied","Data":"b2951ac7fcfa01a10676be91ef4c6b427ccbdc7cdbc74ab6395a70f29b2a42d9"}
Sep 11 12:47:39.753941 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:39.753920    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 12:47:39.754332 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:39.754309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" event={"ID":"8ad4f65c-09c8-4c72-969f-4ac3834c855d","Type":"ContainerStarted","Data":"a95954f356230e134d4147844fd8abef94cf29e1b992cd00f90a48a9ec077093"}
Sep 11 12:47:39.754851 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:39.754835    2564 scope.go:117] "RemoveContainer" containerID="be0d16b676918de951b43c4ea30e8d8633109054032b33d759adc60fe3b78112"
Sep 11 12:47:40.482812 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:40.478343    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:40.482812 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:40.478468    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:40.760684 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:40.760606    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 12:47:40.761008 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:40.760987    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" event={"ID":"8ad4f65c-09c8-4c72-969f-4ac3834c855d","Type":"ContainerStarted","Data":"4cc5a8c25e8a4a0fe3f256350a35062c4d1bebdaaa1ae5ea4986a7b7e6d7f5cc"}
Sep 11 12:47:40.761231 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:40.761211    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:40.761360 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:40.761274    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:40.761360 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:40.761319    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:40.776154 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:40.776125    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:40.777439 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:40.777419    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:47:40.788686 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:40.788613    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx" podStartSLOduration=7.719041238 podStartE2EDuration="27.788589784s" totalImagesPullingTime="20.069548546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:40.787175146 +0000 UTC m=+28.849482103" watchObservedRunningTime="2026-09-11 12:47:40.788589784 +0000 UTC m=+28.850896742"
Sep 11 12:47:41.235533 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:41.235323    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pdqx9"]
Sep 11 12:47:41.235726 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:41.235690    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:41.235809 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:41.235774    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:41.238478 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:41.238448    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2btsm"]
Sep 11 12:47:41.238608 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:41.238593    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:41.238741 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:41.238719    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:41.239164 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:41.239131    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4f7x7"]
Sep 11 12:47:41.239265 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:41.239237    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:41.239363 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:41.239343    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:41.765332 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:41.765293    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2a220c7-9039-4263-b46c-0468f32f4d74" containerID="e65a664daaa88802d643b0329080826fc1aeecd2b59f750daceae193d708ae4d" exitCode=0
Sep 11 12:47:41.765846 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:41.765381    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsq92" event={"ID":"d2a220c7-9039-4263-b46c-0468f32f4d74","Type":"ContainerDied","Data":"e65a664daaa88802d643b0329080826fc1aeecd2b59f750daceae193d708ae4d"}
Sep 11 12:47:42.477717 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:42.477688    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:42.477901 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:42.477688    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:42.477901 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:42.477785    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:42.477901 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:42.477848    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:43.477147 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:43.477102    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:43.477543 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:43.477229    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:43.776025 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:43.775991    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2a220c7-9039-4263-b46c-0468f32f4d74" containerID="119e9d30b7a403a2c5a84fe1319f70a595495a7aa9b4ad628f5cca0ba6f0e139" exitCode=0
Sep 11 12:47:43.776186 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:43.776046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsq92" event={"ID":"d2a220c7-9039-4263-b46c-0468f32f4d74","Type":"ContainerDied","Data":"119e9d30b7a403a2c5a84fe1319f70a595495a7aa9b4ad628f5cca0ba6f0e139"}
Sep 11 12:47:44.477502 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:44.477461    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:44.477968 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:44.477473    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:44.477968 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:44.477585    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:44.477968 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:44.477728    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:45.128156 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:45.128113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:45.128351 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:45.128247    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:45.128351 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:45.128311    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs podName:7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028 nodeName:}" failed. No retries permitted until 2026-09-11 12:48:17.12829395 +0000 UTC m=+65.190600888 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs") pod "network-metrics-daemon-4f7x7" (UID: "7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:47:45.228725 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:45.228690    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:45.228925 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:45.228896    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:45.228992 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:45.228979    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret podName:b32090af-80e4-4ac6-b9b2-da62336a920f nodeName:}" failed. No retries permitted until 2026-09-11 12:48:17.228958386 +0000 UTC m=+65.291265327 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret") pod "global-pull-secret-syncer-2btsm" (UID: "b32090af-80e4-4ac6-b9b2-da62336a920f") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:47:45.329815 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:45.329790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgxb\" (UniqueName: \"kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb\") pod \"network-check-target-pdqx9\" (UID: \"5015a5c1-bdb4-47a0-9ad8-325512861d85\") " pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:45.329975 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:45.329960    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:47:45.330028 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:45.329979    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:47:45.330028 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:45.329989    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qsgxb for pod openshift-network-diagnostics/network-check-target-pdqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:45.330095 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:45.330044    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb podName:5015a5c1-bdb4-47a0-9ad8-325512861d85 nodeName:}" failed. No retries permitted until 2026-09-11 12:48:17.330031223 +0000 UTC m=+65.392338163 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsgxb" (UniqueName: "kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb") pod "network-check-target-pdqx9" (UID: "5015a5c1-bdb4-47a0-9ad8-325512861d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:47:45.476863 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:45.476827    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:45.477045 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:45.476951    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2btsm" podUID="b32090af-80e4-4ac6-b9b2-da62336a920f"
Sep 11 12:47:46.476872 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:46.476839    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:46.477394 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:46.476839    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:46.477394 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:46.476993    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:47:46.477394 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:46.477043    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdqx9" podUID="5015a5c1-bdb4-47a0-9ad8-325512861d85"
Sep 11 12:47:47.232331 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.232294    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-55.ec2.internal" event="NodeReady"
Sep 11 12:47:47.232511 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.232437    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 12:47:47.267110 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.267076    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bc49674fd-2hgfz"]
Sep 11 12:47:47.269892 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.269860    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.272304 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.272277    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wqf2z\""
Sep 11 12:47:47.272304 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.272278    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 12:47:47.272520 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.272446    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 12:47:47.272723 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.272707    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 12:47:47.279147 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.279124    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rt827"]
Sep 11 12:47:47.279556 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.279533    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 12:47:47.281362 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.281343    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nlctv"]
Sep 11 12:47:47.281484 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.281468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.283285 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.283264    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nlctv"
Sep 11 12:47:47.283861 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.283842    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2v8d4\""
Sep 11 12:47:47.283969 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.283867    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 12:47:47.283969 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.283952    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 12:47:47.285483 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.285463    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ss7sh\""
Sep 11 12:47:47.286029 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.286008    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 12:47:47.286315 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.286296    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 12:47:47.286555 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.286533    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bc49674fd-2hgfz"]
Sep 11 12:47:47.286555 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.286549    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 12:47:47.290934 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.290910    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nlctv"]
Sep 11 12:47:47.294729 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.294705    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rt827"]
Sep 11 12:47:47.341108 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341067    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-certificates\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.341287 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-trusted-ca\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.341287 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341150    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-bound-sa-token\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.341287 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341177    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kslxp\" (UniqueName: \"kubernetes.io/projected/53e588b7-21cd-4c00-bfb7-68002b45b7a0-kube-api-access-kslxp\") pod \"ingress-canary-nlctv\" (UID: \"53e588b7-21cd-4c00-bfb7-68002b45b7a0\") " pod="openshift-ingress-canary/ingress-canary-nlctv"
Sep 11 12:47:47.341287 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.341287 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341274    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bhj7\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-kube-api-access-7bhj7\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.341561 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341295    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-config-volume\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.341561 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lvk5\" (UniqueName: \"kubernetes.io/projected/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-kube-api-access-5lvk5\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.341561 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341339    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-image-registry-private-configuration\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.341561 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/284ed70f-dcfc-48d9-aa7b-b323e18955de-ca-trust-extracted\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.341561 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341385    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.341561 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53e588b7-21cd-4c00-bfb7-68002b45b7a0-cert\") pod \"ingress-canary-nlctv\" (UID: \"53e588b7-21cd-4c00-bfb7-68002b45b7a0\") " pod="openshift-ingress-canary/ingress-canary-nlctv"
Sep 11 12:47:47.341561 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-tmp-dir\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.341561 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.341552    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-installation-pull-secrets\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.442223 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-image-registry-private-configuration\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.442416 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/284ed70f-dcfc-48d9-aa7b-b323e18955de-ca-trust-extracted\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.442416 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442258    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.442416 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53e588b7-21cd-4c00-bfb7-68002b45b7a0-cert\") pod \"ingress-canary-nlctv\" (UID: \"53e588b7-21cd-4c00-bfb7-68002b45b7a0\") " pod="openshift-ingress-canary/ingress-canary-nlctv"
Sep 11 12:47:47.442416 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-tmp-dir\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.442416 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442371    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-installation-pull-secrets\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.442416 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-certificates\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.442733 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:47.442417    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 12:47:47.442733 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442432    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-trusted-ca\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.442733 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:47.442439    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc49674fd-2hgfz: secret "image-registry-tls" not found
Sep 11 12:47:47.442733 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-bound-sa-token\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.442733 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:47.442499    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls podName:284ed70f-dcfc-48d9-aa7b-b323e18955de nodeName:}" failed. No retries permitted until 2026-09-11 12:47:47.942478234 +0000 UTC m=+36.004785185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls") pod "image-registry-5bc49674fd-2hgfz" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de") : secret "image-registry-tls" not found
Sep 11 12:47:47.442733 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kslxp\" (UniqueName: \"kubernetes.io/projected/53e588b7-21cd-4c00-bfb7-68002b45b7a0-kube-api-access-kslxp\") pod \"ingress-canary-nlctv\" (UID: \"53e588b7-21cd-4c00-bfb7-68002b45b7a0\") " pod="openshift-ingress-canary/ingress-canary-nlctv"
Sep 11 12:47:47.442733 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.442733 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bhj7\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-kube-api-access-7bhj7\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.442733 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-config-volume\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.442733 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442631    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lvk5\" (UniqueName: \"kubernetes.io/projected/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-kube-api-access-5lvk5\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.443218 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:47.442745    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:47:47.443218 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:47.442818    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls podName:4b04ebea-779c-4b76-9ebb-d66ffa34ee02 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:47.942791147 +0000 UTC m=+36.005098094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls") pod "dns-default-rt827" (UID: "4b04ebea-779c-4b76-9ebb-d66ffa34ee02") : secret "dns-default-metrics-tls" not found
Sep 11 12:47:47.443218 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.442984    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-tmp-dir\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.443364 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.443336    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/284ed70f-dcfc-48d9-aa7b-b323e18955de-ca-trust-extracted\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.443698 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.443621    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-certificates\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.443698 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.443624    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-config-volume\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.444025 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.444005    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-trusted-ca\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.447136 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.446988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53e588b7-21cd-4c00-bfb7-68002b45b7a0-cert\") pod \"ingress-canary-nlctv\" (UID: \"53e588b7-21cd-4c00-bfb7-68002b45b7a0\") " pod="openshift-ingress-canary/ingress-canary-nlctv"
Sep 11 12:47:47.447251 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.446989    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-installation-pull-secrets\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.447251 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.447021    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-image-registry-private-configuration\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.453819 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.453770    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lvk5\" (UniqueName: \"kubernetes.io/projected/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-kube-api-access-5lvk5\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.453819 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.453790    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-bound-sa-token\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.453997 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.453973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kslxp\" (UniqueName: \"kubernetes.io/projected/53e588b7-21cd-4c00-bfb7-68002b45b7a0-kube-api-access-kslxp\") pod \"ingress-canary-nlctv\" (UID: \"53e588b7-21cd-4c00-bfb7-68002b45b7a0\") " pod="openshift-ingress-canary/ingress-canary-nlctv"
Sep 11 12:47:47.454222 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.454197    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bhj7\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-kube-api-access-7bhj7\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.477581 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.477545    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:47:47.480389 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.480361    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 12:47:47.600469 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.600429    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nlctv"
Sep 11 12:47:47.946945 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.946907    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:47.946945 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:47.946957    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:47.947183 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:47.947095    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 12:47:47.947183 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:47.947095    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:47:47.947282 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:47.947184    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls podName:4b04ebea-779c-4b76-9ebb-d66ffa34ee02 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:48.947159154 +0000 UTC m=+37.009466102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls") pod "dns-default-rt827" (UID: "4b04ebea-779c-4b76-9ebb-d66ffa34ee02") : secret "dns-default-metrics-tls" not found
Sep 11 12:47:47.947282 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:47.947108    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc49674fd-2hgfz: secret "image-registry-tls" not found
Sep 11 12:47:47.947386 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:47.947292    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls podName:284ed70f-dcfc-48d9-aa7b-b323e18955de nodeName:}" failed. No retries permitted until 2026-09-11 12:47:48.947279703 +0000 UTC m=+37.009586638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls") pod "image-registry-5bc49674fd-2hgfz" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de") : secret "image-registry-tls" not found
Sep 11 12:47:48.199271 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.199179    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nlctv"]
Sep 11 12:47:48.203364 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:47:48.203338    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e588b7_21cd_4c00_bfb7_68002b45b7a0.slice/crio-a23646bca42c06099d6f0148a9988d9d07c6ac4077e8157d2798e4392455d961 WatchSource:0}: Error finding container a23646bca42c06099d6f0148a9988d9d07c6ac4077e8157d2798e4392455d961: Status 404 returned error can't find the container with id a23646bca42c06099d6f0148a9988d9d07c6ac4077e8157d2798e4392455d961
Sep 11 12:47:48.477247 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.477149    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:47:48.477412 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.477149    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:47:48.480152 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.480127    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 12:47:48.480813 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.480711    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 12:47:48.480813 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.480722    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 12:47:48.481259 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.481236    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9kdd4\""
Sep 11 12:47:48.481259 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.481257    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-86zcf\""
Sep 11 12:47:48.788304 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.788223    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nlctv" event={"ID":"53e588b7-21cd-4c00-bfb7-68002b45b7a0","Type":"ContainerStarted","Data":"a23646bca42c06099d6f0148a9988d9d07c6ac4077e8157d2798e4392455d961"}
Sep 11 12:47:48.790906 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.790879    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2a220c7-9039-4263-b46c-0468f32f4d74" containerID="ffdfbdd65b33545764e3fa0c7c51dea358135a1ca475d4a2940a3e9a76bb0f84" exitCode=0
Sep 11 12:47:48.791044 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.790942    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsq92" event={"ID":"d2a220c7-9039-4263-b46c-0468f32f4d74","Type":"ContainerDied","Data":"ffdfbdd65b33545764e3fa0c7c51dea358135a1ca475d4a2940a3e9a76bb0f84"}
Sep 11 12:47:48.954967 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.954932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:48.955147 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:48.955058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:48.955147 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:48.955112    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 12:47:48.955147 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:48.955137    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc49674fd-2hgfz: secret "image-registry-tls" not found
Sep 11 12:47:48.955273 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:48.955172    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:47:48.955273 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:48.955208    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls podName:284ed70f-dcfc-48d9-aa7b-b323e18955de nodeName:}" failed. No retries permitted until 2026-09-11 12:47:50.955185531 +0000 UTC m=+39.017492484 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls") pod "image-registry-5bc49674fd-2hgfz" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de") : secret "image-registry-tls" not found
Sep 11 12:47:48.955273 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:48.955226    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls podName:4b04ebea-779c-4b76-9ebb-d66ffa34ee02 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:50.955217597 +0000 UTC m=+39.017524534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls") pod "dns-default-rt827" (UID: "4b04ebea-779c-4b76-9ebb-d66ffa34ee02") : secret "dns-default-metrics-tls" not found
Sep 11 12:47:49.797702 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:49.797658    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2a220c7-9039-4263-b46c-0468f32f4d74" containerID="3c5a8c69c05ec8e46cea6ea46649ef4a82561314366466d45e490f00ad390d7c" exitCode=0
Sep 11 12:47:49.797702 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:49.797684    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsq92" event={"ID":"d2a220c7-9039-4263-b46c-0468f32f4d74","Type":"ContainerDied","Data":"3c5a8c69c05ec8e46cea6ea46649ef4a82561314366466d45e490f00ad390d7c"}
Sep 11 12:47:50.803410 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:50.803309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsq92" event={"ID":"d2a220c7-9039-4263-b46c-0468f32f4d74","Type":"ContainerStarted","Data":"fdf391b6401d86b1e4dc2768cdc4a747955f04968cbdff1a23ef64b6f17c1063"}
Sep 11 12:47:50.824891 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:50.824836    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jsq92" podStartSLOduration=7.094361875 podStartE2EDuration="37.824821215s" totalImagesPullingTime="30.73045934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:50.823633246 +0000 UTC m=+38.885940203" watchObservedRunningTime="2026-09-11 12:47:50.824821215 +0000 UTC m=+38.887128174"
Sep 11 12:47:50.973429 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:50.973393    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:50.973581 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:50.973445    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:50.973581 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:50.973544    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 12:47:50.973581 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:50.973557    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc49674fd-2hgfz: secret "image-registry-tls" not found
Sep 11 12:47:50.973721 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:50.973540    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:47:50.973721 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:50.973618    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls podName:284ed70f-dcfc-48d9-aa7b-b323e18955de nodeName:}" failed. No retries permitted until 2026-09-11 12:47:54.973601766 +0000 UTC m=+43.035908702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls") pod "image-registry-5bc49674fd-2hgfz" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de") : secret "image-registry-tls" not found
Sep 11 12:47:50.973721 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:50.973687    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls podName:4b04ebea-779c-4b76-9ebb-d66ffa34ee02 nodeName:}" failed. No retries permitted until 2026-09-11 12:47:54.973675537 +0000 UTC m=+43.035982487 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls") pod "dns-default-rt827" (UID: "4b04ebea-779c-4b76-9ebb-d66ffa34ee02") : secret "dns-default-metrics-tls" not found
Sep 11 12:47:51.806964 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:51.806921    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nlctv" event={"ID":"53e588b7-21cd-4c00-bfb7-68002b45b7a0","Type":"ContainerStarted","Data":"50bff4b376fcf84b8db35646d656f1d04f91eca9d2bbe1ba30c59d3e97d03e27"}
Sep 11 12:47:51.824953 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:51.824906    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nlctv" podStartSLOduration=2.144544421 podStartE2EDuration="4.824890388s" totalImagesPullingTime="2.680345967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:47:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:47:51.823742242 +0000 UTC m=+39.886049199" watchObservedRunningTime="2026-09-11 12:47:51.824890388 +0000 UTC m=+39.887197345"
Sep 11 12:47:55.002666 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:55.002601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:47:55.002666 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:47:55.002673    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:47:55.003109 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:55.002764    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 12:47:55.003109 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:55.002776    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc49674fd-2hgfz: secret "image-registry-tls" not found
Sep 11 12:47:55.003109 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:55.002764    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:47:55.003109 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:55.002864    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls podName:284ed70f-dcfc-48d9-aa7b-b323e18955de nodeName:}" failed. No retries permitted until 2026-09-11 12:48:03.002850662 +0000 UTC m=+51.065157598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls") pod "image-registry-5bc49674fd-2hgfz" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de") : secret "image-registry-tls" not found
Sep 11 12:47:55.003109 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:47:55.002886    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls podName:4b04ebea-779c-4b76-9ebb-d66ffa34ee02 nodeName:}" failed. No retries permitted until 2026-09-11 12:48:03.002873293 +0000 UTC m=+51.065180228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls") pod "dns-default-rt827" (UID: "4b04ebea-779c-4b76-9ebb-d66ffa34ee02") : secret "dns-default-metrics-tls" not found
Sep 11 12:48:03.066189 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:03.066148    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:48:03.066629 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:03.066198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:48:03.066629 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:03.066300    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:48:03.066629 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:03.066318    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 12:48:03.066629 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:03.066328    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc49674fd-2hgfz: secret "image-registry-tls" not found
Sep 11 12:48:03.066629 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:03.066375    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls podName:4b04ebea-779c-4b76-9ebb-d66ffa34ee02 nodeName:}" failed. No retries permitted until 2026-09-11 12:48:19.06635834 +0000 UTC m=+67.128665276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls") pod "dns-default-rt827" (UID: "4b04ebea-779c-4b76-9ebb-d66ffa34ee02") : secret "dns-default-metrics-tls" not found
Sep 11 12:48:03.066629 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:03.066389    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls podName:284ed70f-dcfc-48d9-aa7b-b323e18955de nodeName:}" failed. No retries permitted until 2026-09-11 12:48:19.06638244 +0000 UTC m=+67.128689376 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls") pod "image-registry-5bc49674fd-2hgfz" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de") : secret "image-registry-tls" not found
Sep 11 12:48:12.789832 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:12.789796    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dscjx"
Sep 11 12:48:17.169208 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.169156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:48:17.171867 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.171848    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 12:48:17.180020 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:17.179980    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 12:48:17.180160 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:17.180060    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs podName:7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028 nodeName:}" failed. No retries permitted until 2026-09-11 12:49:21.180042606 +0000 UTC m=+129.242349541 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs") pod "network-metrics-daemon-4f7x7" (UID: "7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028") : secret "metrics-daemon-secret" not found
Sep 11 12:48:17.270039 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.269989    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:48:17.272704 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.272686    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 12:48:17.283786 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.283757    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b32090af-80e4-4ac6-b9b2-da62336a920f-original-pull-secret\") pod \"global-pull-secret-syncer-2btsm\" (UID: \"b32090af-80e4-4ac6-b9b2-da62336a920f\") " pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:48:17.371020 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.370981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgxb\" (UniqueName: \"kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb\") pod \"network-check-target-pdqx9\" (UID: \"5015a5c1-bdb4-47a0-9ad8-325512861d85\") " pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:48:17.373555 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.373533    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 12:48:17.383943 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.383916    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 12:48:17.394147 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.394115    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsgxb\" (UniqueName: \"kubernetes.io/projected/5015a5c1-bdb4-47a0-9ad8-325512861d85-kube-api-access-qsgxb\") pod \"network-check-target-pdqx9\" (UID: \"5015a5c1-bdb4-47a0-9ad8-325512861d85\") " pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:48:17.488250 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.488151    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2btsm"
Sep 11 12:48:17.595019 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.594985    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9kdd4\""
Sep 11 12:48:17.603093 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.603068    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:48:17.754177 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.754083    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2btsm"]
Sep 11 12:48:17.757895 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:48:17.757866    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32090af_80e4_4ac6_b9b2_da62336a920f.slice/crio-8cd0d2053983177e50e7359bc86b25d119f9f02f399bc851f547064ef8b8a65c WatchSource:0}: Error finding container 8cd0d2053983177e50e7359bc86b25d119f9f02f399bc851f547064ef8b8a65c: Status 404 returned error can't find the container with id 8cd0d2053983177e50e7359bc86b25d119f9f02f399bc851f547064ef8b8a65c
Sep 11 12:48:17.869798 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.869765    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pdqx9"]
Sep 11 12:48:17.871435 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:17.871405    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2btsm" event={"ID":"b32090af-80e4-4ac6-b9b2-da62336a920f","Type":"ContainerStarted","Data":"8cd0d2053983177e50e7359bc86b25d119f9f02f399bc851f547064ef8b8a65c"}
Sep 11 12:48:17.872665 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:48:17.872632    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5015a5c1_bdb4_47a0_9ad8_325512861d85.slice/crio-473d82c900fcf7f2cd3b48eb6d1a51492a7395ede79739497409b4620a0e2e32 WatchSource:0}: Error finding container 473d82c900fcf7f2cd3b48eb6d1a51492a7395ede79739497409b4620a0e2e32: Status 404 returned error can't find the container with id 473d82c900fcf7f2cd3b48eb6d1a51492a7395ede79739497409b4620a0e2e32
Sep 11 12:48:18.875465 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:18.875423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pdqx9" event={"ID":"5015a5c1-bdb4-47a0-9ad8-325512861d85","Type":"ContainerStarted","Data":"473d82c900fcf7f2cd3b48eb6d1a51492a7395ede79739497409b4620a0e2e32"}
Sep 11 12:48:19.086062 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:19.086019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:48:19.086256 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:19.086089    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:48:19.086256 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:19.086165    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:48:19.086256 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:19.086235    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 12:48:19.086256 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:19.086249    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc49674fd-2hgfz: secret "image-registry-tls" not found
Sep 11 12:48:19.086470 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:19.086250    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls podName:4b04ebea-779c-4b76-9ebb-d66ffa34ee02 nodeName:}" failed. No retries permitted until 2026-09-11 12:48:51.086225989 +0000 UTC m=+99.148532948 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls") pod "dns-default-rt827" (UID: "4b04ebea-779c-4b76-9ebb-d66ffa34ee02") : secret "dns-default-metrics-tls" not found
Sep 11 12:48:19.086470 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:19.086310    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls podName:284ed70f-dcfc-48d9-aa7b-b323e18955de nodeName:}" failed. No retries permitted until 2026-09-11 12:48:51.086294218 +0000 UTC m=+99.148601155 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls") pod "image-registry-5bc49674fd-2hgfz" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de") : secret "image-registry-tls" not found
Sep 11 12:48:22.887837 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:22.887737    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2btsm" event={"ID":"b32090af-80e4-4ac6-b9b2-da62336a920f","Type":"ContainerStarted","Data":"21056bb47868b7e47dc8de8ac4b24e57b4a9d0a0c2fe273a0da634b88b1a4321"}
Sep 11 12:48:22.889329 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:22.889303    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pdqx9" event={"ID":"5015a5c1-bdb4-47a0-9ad8-325512861d85","Type":"ContainerStarted","Data":"61a39c9de2d4f57b4f73c72cd566cfc53ee86ded6573ba3f9c7a7301932406bb"}
Sep 11 12:48:22.889460 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:22.889437    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:48:22.903777 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:22.903722    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2btsm" podStartSLOduration=65.02321675 podStartE2EDuration="1m9.903704888s" totalImagesPullingTime="4.880488138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:48:22.903371317 +0000 UTC m=+70.965678275" watchObservedRunningTime="2026-09-11 12:48:22.903704888 +0000 UTC m=+70.966011845"
Sep 11 12:48:22.919245 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:22.919193    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pdqx9" podStartSLOduration=65.157221475 podStartE2EDuration="1m9.919174562s" totalImagesPullingTime="4.761953087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:48:22.918478548 +0000 UTC m=+70.980785519" watchObservedRunningTime="2026-09-11 12:48:22.919174562 +0000 UTC m=+70.981481519"
Sep 11 12:48:26.737212 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.737177    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"]
Sep 11 12:48:26.739083 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.739065    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"
Sep 11 12:48:26.742741 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.742709    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 12:48:26.742741 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.742726    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 12:48:26.742741 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.742726    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 12:48:26.742993 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.742730    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 11 12:48:26.742993 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.742730    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-54fcn\""
Sep 11 12:48:26.750867 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.750835    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"]
Sep 11 12:48:26.844921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.844868    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5vcq\" (UniqueName: \"kubernetes.io/projected/16fd2df8-0449-4832-95ed-37bbadba1e0f-kube-api-access-n5vcq\") pod \"managed-serviceaccount-addon-agent-94ff845fd-x8dfh\" (UID: \"16fd2df8-0449-4832-95ed-37bbadba1e0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"
Sep 11 12:48:26.845114 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.844946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16fd2df8-0449-4832-95ed-37bbadba1e0f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-94ff845fd-x8dfh\" (UID: \"16fd2df8-0449-4832-95ed-37bbadba1e0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"
Sep 11 12:48:26.945403 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.945355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5vcq\" (UniqueName: \"kubernetes.io/projected/16fd2df8-0449-4832-95ed-37bbadba1e0f-kube-api-access-n5vcq\") pod \"managed-serviceaccount-addon-agent-94ff845fd-x8dfh\" (UID: \"16fd2df8-0449-4832-95ed-37bbadba1e0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"
Sep 11 12:48:26.945571 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.945422    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16fd2df8-0449-4832-95ed-37bbadba1e0f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-94ff845fd-x8dfh\" (UID: \"16fd2df8-0449-4832-95ed-37bbadba1e0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"
Sep 11 12:48:26.948063 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.948028    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16fd2df8-0449-4832-95ed-37bbadba1e0f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-94ff845fd-x8dfh\" (UID: \"16fd2df8-0449-4832-95ed-37bbadba1e0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"
Sep 11 12:48:26.954516 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:26.954489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5vcq\" (UniqueName: \"kubernetes.io/projected/16fd2df8-0449-4832-95ed-37bbadba1e0f-kube-api-access-n5vcq\") pod \"managed-serviceaccount-addon-agent-94ff845fd-x8dfh\" (UID: \"16fd2df8-0449-4832-95ed-37bbadba1e0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"
Sep 11 12:48:27.058662 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:27.058557    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"
Sep 11 12:48:27.330896 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:27.330863    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh"]
Sep 11 12:48:27.334557 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:48:27.334527    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16fd2df8_0449_4832_95ed_37bbadba1e0f.slice/crio-435229101ef37bd40178cdb550e0fc29702750dc44ab866ecabcc31d10785877 WatchSource:0}: Error finding container 435229101ef37bd40178cdb550e0fc29702750dc44ab866ecabcc31d10785877: Status 404 returned error can't find the container with id 435229101ef37bd40178cdb550e0fc29702750dc44ab866ecabcc31d10785877
Sep 11 12:48:27.904993 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:27.904946    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh" event={"ID":"16fd2df8-0449-4832-95ed-37bbadba1e0f","Type":"ContainerStarted","Data":"435229101ef37bd40178cdb550e0fc29702750dc44ab866ecabcc31d10785877"}
Sep 11 12:48:29.912811 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:29.912776    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh" event={"ID":"16fd2df8-0449-4832-95ed-37bbadba1e0f","Type":"ContainerStarted","Data":"a3b1a6caee15bf9080bbc1df05e665147a67ced3427a1b6df220c70f216ace62"}
Sep 11 12:48:29.929693 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:29.929625    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh" podStartSLOduration=1.710725258 podStartE2EDuration="3.929609891s" totalImagesPullingTime="2.218884633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:48:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:48:29.929495037 +0000 UTC m=+77.991801996" watchObservedRunningTime="2026-09-11 12:48:29.929609891 +0000 UTC m=+77.991916849"
Sep 11 12:48:51.117680 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:51.117620    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:48:51.118044 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:51.117733    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:48:51.118044 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:51.117776    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 12:48:51.118044 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:51.117795    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc49674fd-2hgfz: secret "image-registry-tls" not found
Sep 11 12:48:51.118044 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:51.117829    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:48:51.118044 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:51.117855    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls podName:284ed70f-dcfc-48d9-aa7b-b323e18955de nodeName:}" failed. No retries permitted until 2026-09-11 12:49:55.117840896 +0000 UTC m=+163.180147832 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls") pod "image-registry-5bc49674fd-2hgfz" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de") : secret "image-registry-tls" not found
Sep 11 12:48:51.118044 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:48:51.117876    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls podName:4b04ebea-779c-4b76-9ebb-d66ffa34ee02 nodeName:}" failed. No retries permitted until 2026-09-11 12:49:55.117865105 +0000 UTC m=+163.180172041 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls") pod "dns-default-rt827" (UID: "4b04ebea-779c-4b76-9ebb-d66ffa34ee02") : secret "dns-default-metrics-tls" not found
Sep 11 12:48:53.894374 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:48:53.894340    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pdqx9"
Sep 11 12:49:21.221572 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:49:21.221521    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:49:21.222083 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:49:21.221694    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 12:49:21.222083 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:49:21.221773    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs podName:7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:23.22175696 +0000 UTC m=+251.284063901 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs") pod "network-metrics-daemon-4f7x7" (UID: "7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028") : secret "metrics-daemon-secret" not found
Sep 11 12:49:50.282134 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:49:50.282083    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz" podUID="284ed70f-dcfc-48d9-aa7b-b323e18955de"
Sep 11 12:49:50.294372 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:49:50.294346    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rt827" podUID="4b04ebea-779c-4b76-9ebb-d66ffa34ee02"
Sep 11 12:49:51.109325 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:49:51.109292    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:49:51.109489 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:49:51.109323    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rt827"
Sep 11 12:49:51.487485 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:49:51.487388    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:49:55.163321 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:49:55.163275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:49:55.163321 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:49:55.163327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") pod \"image-registry-5bc49674fd-2hgfz\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") " pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:49:55.163868 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:49:55.163433    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:49:55.163868 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:49:55.163509    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls podName:4b04ebea-779c-4b76-9ebb-d66ffa34ee02 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:57.163493463 +0000 UTC m=+285.225800399 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls") pod "dns-default-rt827" (UID: "4b04ebea-779c-4b76-9ebb-d66ffa34ee02") : secret "dns-default-metrics-tls" not found
Sep 11 12:49:55.163868 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:49:55.163436    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 12:49:55.163868 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:49:55.163531    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc49674fd-2hgfz: secret "image-registry-tls" not found
Sep 11 12:49:55.163868 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:49:55.163591    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls podName:284ed70f-dcfc-48d9-aa7b-b323e18955de nodeName:}" failed. No retries permitted until 2026-09-11 12:51:57.163577869 +0000 UTC m=+285.225884813 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls") pod "image-registry-5bc49674fd-2hgfz" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de") : secret "image-registry-tls" not found
Sep 11 12:50:04.477061 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:50:04.476963    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:50:30.204060 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:50:30.204026    2564 generic.go:356] "Generic (PLEG): container finished" podID="16fd2df8-0449-4832-95ed-37bbadba1e0f" containerID="a3b1a6caee15bf9080bbc1df05e665147a67ced3427a1b6df220c70f216ace62" exitCode=255
Sep 11 12:50:30.204485 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:50:30.204080    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh" event={"ID":"16fd2df8-0449-4832-95ed-37bbadba1e0f","Type":"ContainerDied","Data":"a3b1a6caee15bf9080bbc1df05e665147a67ced3427a1b6df220c70f216ace62"}
Sep 11 12:50:30.209830 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:50:30.209807    2564 scope.go:117] "RemoveContainer" containerID="a3b1a6caee15bf9080bbc1df05e665147a67ced3427a1b6df220c70f216ace62"
Sep 11 12:50:31.209583 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:50:31.209542    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-94ff845fd-x8dfh" event={"ID":"16fd2df8-0449-4832-95ed-37bbadba1e0f","Type":"ContainerStarted","Data":"e23922de3e75a4507df312c1fe4c57af301e95bc6a97c3742fb21e4b32a0c4a0"}
Sep 11 12:51:20.001656 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.001607    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"]
Sep 11 12:51:20.004449 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.004429    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.011072 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.011050    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 12:51:20.011302 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.011288    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 12:51:20.012112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.012091    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6f7q6\""
Sep 11 12:51:20.012112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.012106    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 12:51:20.012263 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.012199    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 12:51:20.026686 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.026657    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"]
Sep 11 12:51:20.029505 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.029483    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"]
Sep 11 12:51:20.029622 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.029580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:20.033824 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.033794    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 11 12:51:20.033824 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.033802    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 11 12:51:20.034035 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.033851    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ppzsr\""
Sep 11 12:51:20.034139 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.034123    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 11 12:51:20.052459 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.052426    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"]
Sep 11 12:51:20.065062 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.065032    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c859ee38-4f38-46b2-8b8b-ae6345444bc6-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-l2k5w\" (UID: \"c859ee38-4f38-46b2-8b8b-ae6345444bc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.065259 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.065088    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c859ee38-4f38-46b2-8b8b-ae6345444bc6-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-l2k5w\" (UID: \"c859ee38-4f38-46b2-8b8b-ae6345444bc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.065259 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.065228    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2gx8\" (UniqueName: \"kubernetes.io/projected/c859ee38-4f38-46b2-8b8b-ae6345444bc6-kube-api-access-c2gx8\") pod \"kube-storage-version-migrator-operator-6c8f84f897-l2k5w\" (UID: \"c859ee38-4f38-46b2-8b8b-ae6345444bc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.109847 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.109809    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"]
Sep 11 12:51:20.112475 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.112458    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.114389 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.114366    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bm485"]
Sep 11 12:51:20.116904 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.116888    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-qzskx"]
Sep 11 12:51:20.117012 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.116995    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bm485"
Sep 11 12:51:20.118863 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.118840    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zkltw\""
Sep 11 12:51:20.119587 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.119569    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.122838 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.122821    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 12:51:20.124659 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.124626    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 12:51:20.124777 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.124627    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 12:51:20.124777 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.124771    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 12:51:20.124912 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.124862    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 12:51:20.124912 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.124875    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 12:51:20.125034 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.124962    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 12:51:20.125154 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.125135    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 12:51:20.125236 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.125173    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7fvcb\""
Sep 11 12:51:20.125486 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.125451    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5mtcd\""
Sep 11 12:51:20.130110 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.130090    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 12:51:20.132612 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.132593    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"]
Sep 11 12:51:20.136016 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.135996    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bm485"]
Sep 11 12:51:20.141354 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.141330    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-qzskx"]
Sep 11 12:51:20.166527 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.166496    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:20.166719 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.166539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c859ee38-4f38-46b2-8b8b-ae6345444bc6-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-l2k5w\" (UID: \"c859ee38-4f38-46b2-8b8b-ae6345444bc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.166719 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.166603    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c859ee38-4f38-46b2-8b8b-ae6345444bc6-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-l2k5w\" (UID: \"c859ee38-4f38-46b2-8b8b-ae6345444bc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.166719 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.166693    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qttfg\" (UniqueName: \"kubernetes.io/projected/c4485948-2373-425a-8295-5b95217023b2-kube-api-access-qttfg\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:20.166845 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.166720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2gx8\" (UniqueName: \"kubernetes.io/projected/c859ee38-4f38-46b2-8b8b-ae6345444bc6-kube-api-access-c2gx8\") pod \"kube-storage-version-migrator-operator-6c8f84f897-l2k5w\" (UID: \"c859ee38-4f38-46b2-8b8b-ae6345444bc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.167145 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.167127    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c859ee38-4f38-46b2-8b8b-ae6345444bc6-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-l2k5w\" (UID: \"c859ee38-4f38-46b2-8b8b-ae6345444bc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.168759 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.168740    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c859ee38-4f38-46b2-8b8b-ae6345444bc6-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-l2k5w\" (UID: \"c859ee38-4f38-46b2-8b8b-ae6345444bc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.179178 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.179138    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2gx8\" (UniqueName: \"kubernetes.io/projected/c859ee38-4f38-46b2-8b8b-ae6345444bc6-kube-api-access-c2gx8\") pod \"kube-storage-version-migrator-operator-6c8f84f897-l2k5w\" (UID: \"c859ee38-4f38-46b2-8b8b-ae6345444bc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.268043 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.267947    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xph5m\" (UniqueName: \"kubernetes.io/projected/451e1f36-c6f2-46e1-b4e1-cd00428c7216-kube-api-access-xph5m\") pod \"network-check-source-6f8b768455-bm485\" (UID: \"451e1f36-c6f2-46e1-b4e1-cd00428c7216\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bm485"
Sep 11 12:51:20.268043 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.267991    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8583207f-fafa-425c-8d07-cdf8a61083ab-serving-cert\") pod \"service-ca-operator-cf7c776b7-wsg8z\" (UID: \"8583207f-fafa-425c-8d07-cdf8a61083ab\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.268043 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.268015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:20.268314 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.268091    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-tmp\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.268314 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.268128    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8583207f-fafa-425c-8d07-cdf8a61083ab-config\") pod \"service-ca-operator-cf7c776b7-wsg8z\" (UID: \"8583207f-fafa-425c-8d07-cdf8a61083ab\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.268314 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.268156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fdh9\" (UniqueName: \"kubernetes.io/projected/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-kube-api-access-2fdh9\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.268314 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:20.268161    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 12:51:20.268314 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.268202    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-snapshots\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.268314 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:20.268287    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls podName:c4485948-2373-425a-8295-5b95217023b2 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:20.768271355 +0000 UTC m=+248.830578290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2br76" (UID: "c4485948-2373-425a-8295-5b95217023b2") : secret "samples-operator-tls" not found
Sep 11 12:51:20.268532 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.268372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-serving-cert\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.268532 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.268409    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-trusted-ca-bundle\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.268532 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.268434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qttfg\" (UniqueName: \"kubernetes.io/projected/c4485948-2373-425a-8295-5b95217023b2-kube-api-access-qttfg\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:20.268532 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.268519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fbjr\" (UniqueName: \"kubernetes.io/projected/8583207f-fafa-425c-8d07-cdf8a61083ab-kube-api-access-4fbjr\") pod \"service-ca-operator-cf7c776b7-wsg8z\" (UID: \"8583207f-fafa-425c-8d07-cdf8a61083ab\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.268690 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.268545    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-service-ca-bundle\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.280270 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.280236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qttfg\" (UniqueName: \"kubernetes.io/projected/c4485948-2373-425a-8295-5b95217023b2-kube-api-access-qttfg\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:20.313238 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.313192    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"
Sep 11 12:51:20.369148 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369121    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-tmp\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.369261 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8583207f-fafa-425c-8d07-cdf8a61083ab-config\") pod \"service-ca-operator-cf7c776b7-wsg8z\" (UID: \"8583207f-fafa-425c-8d07-cdf8a61083ab\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.369261 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369180    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fdh9\" (UniqueName: \"kubernetes.io/projected/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-kube-api-access-2fdh9\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.369515 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369488    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-snapshots\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.369592 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-serving-cert\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.369592 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369575    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-tmp\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.369703 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369633    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-trusted-ca-bundle\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.369747 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369714    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fbjr\" (UniqueName: \"kubernetes.io/projected/8583207f-fafa-425c-8d07-cdf8a61083ab-kube-api-access-4fbjr\") pod \"service-ca-operator-cf7c776b7-wsg8z\" (UID: \"8583207f-fafa-425c-8d07-cdf8a61083ab\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.369798 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-service-ca-bundle\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.369854 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xph5m\" (UniqueName: \"kubernetes.io/projected/451e1f36-c6f2-46e1-b4e1-cd00428c7216-kube-api-access-xph5m\") pod \"network-check-source-6f8b768455-bm485\" (UID: \"451e1f36-c6f2-46e1-b4e1-cd00428c7216\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bm485"
Sep 11 12:51:20.369854 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8583207f-fafa-425c-8d07-cdf8a61083ab-serving-cert\") pod \"service-ca-operator-cf7c776b7-wsg8z\" (UID: \"8583207f-fafa-425c-8d07-cdf8a61083ab\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.369955 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.369920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8583207f-fafa-425c-8d07-cdf8a61083ab-config\") pod \"service-ca-operator-cf7c776b7-wsg8z\" (UID: \"8583207f-fafa-425c-8d07-cdf8a61083ab\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.370282 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.370152    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-snapshots\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.370487 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.370458    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-service-ca-bundle\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.370697 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.370682    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-trusted-ca-bundle\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.372446 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.372398    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8583207f-fafa-425c-8d07-cdf8a61083ab-serving-cert\") pod \"service-ca-operator-cf7c776b7-wsg8z\" (UID: \"8583207f-fafa-425c-8d07-cdf8a61083ab\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.372542 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.372442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-serving-cert\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.380276 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.380245    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fbjr\" (UniqueName: \"kubernetes.io/projected/8583207f-fafa-425c-8d07-cdf8a61083ab-kube-api-access-4fbjr\") pod \"service-ca-operator-cf7c776b7-wsg8z\" (UID: \"8583207f-fafa-425c-8d07-cdf8a61083ab\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.380389 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.380299    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xph5m\" (UniqueName: \"kubernetes.io/projected/451e1f36-c6f2-46e1-b4e1-cd00428c7216-kube-api-access-xph5m\") pod \"network-check-source-6f8b768455-bm485\" (UID: \"451e1f36-c6f2-46e1-b4e1-cd00428c7216\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bm485"
Sep 11 12:51:20.382617 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.382596    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fdh9\" (UniqueName: \"kubernetes.io/projected/cdad3f72-dc2b-423e-8cd5-cbb2445667fb-kube-api-access-2fdh9\") pod \"insights-operator-5844b856db-qzskx\" (UID: \"cdad3f72-dc2b-423e-8cd5-cbb2445667fb\") " pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.422080 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.422029    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"
Sep 11 12:51:20.429002 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.428978    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bm485"
Sep 11 12:51:20.434989 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.434960    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-qzskx"
Sep 11 12:51:20.585684 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.585659    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w"]
Sep 11 12:51:20.588001 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:51:20.587976    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc859ee38_4f38_46b2_8b8b_ae6345444bc6.slice/crio-b78e919c5dce226e1dd41249b64cae87ab1a9c187f164a679120d14a81803b23 WatchSource:0}: Error finding container b78e919c5dce226e1dd41249b64cae87ab1a9c187f164a679120d14a81803b23: Status 404 returned error can't find the container with id b78e919c5dce226e1dd41249b64cae87ab1a9c187f164a679120d14a81803b23
Sep 11 12:51:20.692462 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.692431    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z"]
Sep 11 12:51:20.695869 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:51:20.695844    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8583207f_fafa_425c_8d07_cdf8a61083ab.slice/crio-0bec1c549d21931c4f78ea496d89df281c5ce21317b9b07e0a30986fea940d4f WatchSource:0}: Error finding container 0bec1c549d21931c4f78ea496d89df281c5ce21317b9b07e0a30986fea940d4f: Status 404 returned error can't find the container with id 0bec1c549d21931c4f78ea496d89df281c5ce21317b9b07e0a30986fea940d4f
Sep 11 12:51:20.704010 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.703985    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bm485"]
Sep 11 12:51:20.706649 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:51:20.706616    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod451e1f36_c6f2_46e1_b4e1_cd00428c7216.slice/crio-3d26bd3c89112bbfdf390b29ee75b2fdd37557116c78ccedf36284a72a0e862f WatchSource:0}: Error finding container 3d26bd3c89112bbfdf390b29ee75b2fdd37557116c78ccedf36284a72a0e862f: Status 404 returned error can't find the container with id 3d26bd3c89112bbfdf390b29ee75b2fdd37557116c78ccedf36284a72a0e862f
Sep 11 12:51:20.726820 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.724556    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-qzskx"]
Sep 11 12:51:20.772660 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:20.772608    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:20.772986 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:20.772960    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 12:51:20.773060 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:20.773047    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls podName:c4485948-2373-425a-8295-5b95217023b2 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:21.773025537 +0000 UTC m=+249.835332492 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2br76" (UID: "c4485948-2373-425a-8295-5b95217023b2") : secret "samples-operator-tls" not found
Sep 11 12:51:21.351343 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:21.350966    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bm485" event={"ID":"451e1f36-c6f2-46e1-b4e1-cd00428c7216","Type":"ContainerStarted","Data":"a8e1efb68f1770e20fc68efab665be7ca2f75bf9e18480f0076cefacac66d88d"}
Sep 11 12:51:21.351343 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:21.351038    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bm485" event={"ID":"451e1f36-c6f2-46e1-b4e1-cd00428c7216","Type":"ContainerStarted","Data":"3d26bd3c89112bbfdf390b29ee75b2fdd37557116c78ccedf36284a72a0e862f"}
Sep 11 12:51:21.352898 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:21.352862    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w" event={"ID":"c859ee38-4f38-46b2-8b8b-ae6345444bc6","Type":"ContainerStarted","Data":"b78e919c5dce226e1dd41249b64cae87ab1a9c187f164a679120d14a81803b23"}
Sep 11 12:51:21.354302 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:21.354270    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qzskx" event={"ID":"cdad3f72-dc2b-423e-8cd5-cbb2445667fb","Type":"ContainerStarted","Data":"b95b82ec7f24335b6195502c9955ccfaafac07a0fc9131408e20cba9002b3530"}
Sep 11 12:51:21.356192 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:21.356165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z" event={"ID":"8583207f-fafa-425c-8d07-cdf8a61083ab","Type":"ContainerStarted","Data":"0bec1c549d21931c4f78ea496d89df281c5ce21317b9b07e0a30986fea940d4f"}
Sep 11 12:51:21.379925 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:21.379805    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bm485" podStartSLOduration=1.3797813159999999 podStartE2EDuration="1.379781316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:51:21.378803486 +0000 UTC m=+249.441110445" watchObservedRunningTime="2026-09-11 12:51:21.379781316 +0000 UTC m=+249.442088274"
Sep 11 12:51:21.779667 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:21.779552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:21.779836 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:21.779804    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 12:51:21.779908 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:21.779879    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls podName:c4485948-2373-425a-8295-5b95217023b2 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:23.779859173 +0000 UTC m=+251.842166116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2br76" (UID: "c4485948-2373-425a-8295-5b95217023b2") : secret "samples-operator-tls" not found
Sep 11 12:51:23.292782 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:23.292737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:51:23.293214 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:23.292916    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 12:51:23.293214 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:23.292997    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs podName:7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028 nodeName:}" failed. No retries permitted until 2026-09-11 12:53:25.292974484 +0000 UTC m=+373.355281437 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs") pod "network-metrics-daemon-4f7x7" (UID: "7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028") : secret "metrics-daemon-secret" not found
Sep 11 12:51:23.796378 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:23.796344    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:23.796530 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:23.796513    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 12:51:23.796612 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:23.796600    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls podName:c4485948-2373-425a-8295-5b95217023b2 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:27.796577956 +0000 UTC m=+255.858884912 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2br76" (UID: "c4485948-2373-425a-8295-5b95217023b2") : secret "samples-operator-tls" not found
Sep 11 12:51:24.371198 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:24.371163    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w" event={"ID":"c859ee38-4f38-46b2-8b8b-ae6345444bc6","Type":"ContainerStarted","Data":"c668a2383ba8775fc2783d6a3c072f3a20109f904c3c9c0c4842728f3664ffeb"}
Sep 11 12:51:24.372749 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:24.372722    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qzskx" event={"ID":"cdad3f72-dc2b-423e-8cd5-cbb2445667fb","Type":"ContainerStarted","Data":"d7d3546e5b66daeae73e71eab75880e679c56a4bfee1c421229706c2eb669d69"}
Sep 11 12:51:24.374226 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:24.374199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z" event={"ID":"8583207f-fafa-425c-8d07-cdf8a61083ab","Type":"ContainerStarted","Data":"6ac1f41009ba0d07bdac30d57c85ac617ec904a7cacf472f6390d58301533737"}
Sep 11 12:51:24.394021 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:24.393964    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w" podStartSLOduration=2.240085597 podStartE2EDuration="5.393946386s" totalImagesPullingTime="3.153860789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:51:24.392174823 +0000 UTC m=+252.454481781" watchObservedRunningTime="2026-09-11 12:51:24.393946386 +0000 UTC m=+252.456253382"
Sep 11 12:51:24.414434 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:24.414382    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z" podStartSLOduration=1.363268451 podStartE2EDuration="4.414360685s" totalImagesPullingTime="3.051092234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:51:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:51:24.414258577 +0000 UTC m=+252.476565536" watchObservedRunningTime="2026-09-11 12:51:24.414360685 +0000 UTC m=+252.476667642"
Sep 11 12:51:24.436440 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:24.436370    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-qzskx" podStartSLOduration=1.4163109440000001 podStartE2EDuration="4.436347128s" totalImagesPullingTime="3.020036184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:51:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:51:24.43599818 +0000 UTC m=+252.498305138" watchObservedRunningTime="2026-09-11 12:51:24.436347128 +0000 UTC m=+252.498654090"
Sep 11 12:51:25.011219 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.011188    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq"]
Sep 11 12:51:25.016545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.016515    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq"
Sep 11 12:51:25.021127 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.021098    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 12:51:25.021127 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.021119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-cx2hn\""
Sep 11 12:51:25.021328 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.021146    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 12:51:25.043582 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.043546    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq"]
Sep 11 12:51:25.108530 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.108502    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w42r7\" (UniqueName: \"kubernetes.io/projected/1f93c707-6182-4d68-a7e0-4e6260ff41c5-kube-api-access-w42r7\") pod \"migrator-8b4d697c5-fsbsq\" (UID: \"1f93c707-6182-4d68-a7e0-4e6260ff41c5\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq"
Sep 11 12:51:25.154804 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.154773    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2c55d"]
Sep 11 12:51:25.157892 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.157873    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.169421 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.169399    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 12:51:25.169421 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.169407    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rqv5f\""
Sep 11 12:51:25.169865 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.169848    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 12:51:25.194151 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.194116    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2c55d"]
Sep 11 12:51:25.209261 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.209225    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w42r7\" (UniqueName: \"kubernetes.io/projected/1f93c707-6182-4d68-a7e0-4e6260ff41c5-kube-api-access-w42r7\") pod \"migrator-8b4d697c5-fsbsq\" (UID: \"1f93c707-6182-4d68-a7e0-4e6260ff41c5\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq"
Sep 11 12:51:25.227682 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.227635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w42r7\" (UniqueName: \"kubernetes.io/projected/1f93c707-6182-4d68-a7e0-4e6260ff41c5-kube-api-access-w42r7\") pod \"migrator-8b4d697c5-fsbsq\" (UID: \"1f93c707-6182-4d68-a7e0-4e6260ff41c5\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq"
Sep 11 12:51:25.309798 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.309697    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7glh2\" (UniqueName: \"kubernetes.io/projected/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-api-access-7glh2\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.309958 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.309853    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.309958 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.309939    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8a891b1-0df9-4a78-9936-82eea74ff958-crio-socket\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.310035 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.309966    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8a891b1-0df9-4a78-9936-82eea74ff958-data-volume\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.310035 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.309994    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.325781 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.325742    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq"
Sep 11 12:51:25.411108 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.411074    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.411493 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.411129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8a891b1-0df9-4a78-9936-82eea74ff958-crio-socket\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.411493 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.411147    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8a891b1-0df9-4a78-9936-82eea74ff958-data-volume\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.411493 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.411162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.411493 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:25.411221    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 12:51:25.411493 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:25.411232    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:25.411493 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.411268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8a891b1-0df9-4a78-9936-82eea74ff958-crio-socket\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.411493 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:25.411275    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:25.911260031 +0000 UTC m=+253.973566967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : configmap "kube-rbac-proxy" not found
Sep 11 12:51:25.411493 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:25.411326    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:25.911307489 +0000 UTC m=+253.973614431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:25.411493 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.411386    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7glh2\" (UniqueName: \"kubernetes.io/projected/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-api-access-7glh2\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.411493 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.411493    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8a891b1-0df9-4a78-9936-82eea74ff958-data-volume\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.423249 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.423202    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7glh2\" (UniqueName: \"kubernetes.io/projected/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-api-access-7glh2\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.603888 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.603748    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq"]
Sep 11 12:51:25.606286 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:51:25.606262    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f93c707_6182_4d68_a7e0_4e6260ff41c5.slice/crio-875abfd7edb32c66f04466714e7839b256678c91759c5868cba67fe93c5b3c5c WatchSource:0}: Error finding container 875abfd7edb32c66f04466714e7839b256678c91759c5868cba67fe93c5b3c5c: Status 404 returned error can't find the container with id 875abfd7edb32c66f04466714e7839b256678c91759c5868cba67fe93c5b3c5c
Sep 11 12:51:25.916031 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.915940    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.916231 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:25.916043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:25.916231 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:25.916102    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 12:51:25.916231 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:25.916162    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:25.916231 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:25.916183    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:26.916161549 +0000 UTC m=+254.978468498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : configmap "kube-rbac-proxy" not found
Sep 11 12:51:25.916231 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:25.916203    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:26.916191239 +0000 UTC m=+254.978498175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:26.383374 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:26.383340    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq" event={"ID":"1f93c707-6182-4d68-a7e0-4e6260ff41c5","Type":"ContainerStarted","Data":"875abfd7edb32c66f04466714e7839b256678c91759c5868cba67fe93c5b3c5c"}
Sep 11 12:51:26.743084 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:26.742996    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7lzcr_439fc464-12e7-415c-922c-c2ed1971e596/dns-node-resolver/0.log"
Sep 11 12:51:26.924497 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:26.924450    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:26.924673 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:26.924557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:26.924673 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:26.924605    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:26.924764 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:26.924694    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 12:51:26.924764 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:26.924708    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:28.92469086 +0000 UTC m=+256.986997825 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:26.924764 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:26.924747    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:28.924732203 +0000 UTC m=+256.987039139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : configmap "kube-rbac-proxy" not found
Sep 11 12:51:27.391880 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:27.391840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq" event={"ID":"1f93c707-6182-4d68-a7e0-4e6260ff41c5","Type":"ContainerStarted","Data":"5c6e214a7cb574d7554ef320be98a0dbd843b0ca921c682d0a6e4d8be9a0c865"}
Sep 11 12:51:27.392028 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:27.391907    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq" event={"ID":"1f93c707-6182-4d68-a7e0-4e6260ff41c5","Type":"ContainerStarted","Data":"89de767a0853a083ba9c77abed7aa74bcf64bfa3f299311362d5622285409b3d"}
Sep 11 12:51:27.414367 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:27.414304    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-fsbsq" podStartSLOduration=1.838403003 podStartE2EDuration="3.414286529s" totalImagesPullingTime="1.575883526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:51:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:51:27.414113631 +0000 UTC m=+255.476420589" watchObservedRunningTime="2026-09-11 12:51:27.414286529 +0000 UTC m=+255.476593512"
Sep 11 12:51:27.830516 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:27.830462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:27.830931 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:27.830615    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 12:51:27.830931 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:27.830709    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls podName:c4485948-2373-425a-8295-5b95217023b2 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:35.830693709 +0000 UTC m=+263.893000650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-2br76" (UID: "c4485948-2373-425a-8295-5b95217023b2") : secret "samples-operator-tls" not found
Sep 11 12:51:28.142952 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:28.142872    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zbccg_5d9fee76-abbc-4b67-bfaf-3fe94c1e6353/node-ca/0.log"
Sep 11 12:51:28.939923 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:28.939879    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:28.940326 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:28.939971    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:28.940326 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:28.940007    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 12:51:28.940326 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:28.940066    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:28.940326 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:28.940072    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:32.940058194 +0000 UTC m=+261.002365141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : configmap "kube-rbac-proxy" not found
Sep 11 12:51:28.940326 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:28.940106    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:32.940095473 +0000 UTC m=+261.002402419 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:28.944695 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:28.944669    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nlctv_53e588b7-21cd-4c00-bfb7-68002b45b7a0/serve-healthcheck-canary/0.log"
Sep 11 12:51:32.972854 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:32.972808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:32.973242 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:32.972874    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:32.973242 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:32.972937    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 12:51:32.973242 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:32.972970    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:32.973242 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:32.972996    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:40.9729783 +0000 UTC m=+269.035285242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : configmap "kube-rbac-proxy" not found
Sep 11 12:51:32.973242 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:32.973027    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:40.973014409 +0000 UTC m=+269.035321345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:35.899579 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:35.899488    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:35.901937 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:35.901916    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4485948-2373-425a-8295-5b95217023b2-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-2br76\" (UID: \"c4485948-2373-425a-8295-5b95217023b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:35.938921 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:35.938878    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"
Sep 11 12:51:36.217393 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:36.217284    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76"]
Sep 11 12:51:36.424129 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:36.424099    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76" event={"ID":"c4485948-2373-425a-8295-5b95217023b2","Type":"ContainerStarted","Data":"7e567a63a09353dcee8d48df0cc8cf6e7213f743f89ab10442025bfac718b1df"}
Sep 11 12:51:38.433352 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:38.433311    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76" event={"ID":"c4485948-2373-425a-8295-5b95217023b2","Type":"ContainerStarted","Data":"a407a451998012308a94fce7a6249ed07a745239bff96cd3f9e99f3d21b1b506"}
Sep 11 12:51:38.433792 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:38.433416    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76" event={"ID":"c4485948-2373-425a-8295-5b95217023b2","Type":"ContainerStarted","Data":"bed9b1be78c6f92cff1c2deaec6c5571b475f3d32390055586d126f2b1ac1dd4"}
Sep 11 12:51:38.451400 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:38.451344    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-2br76" podStartSLOduration=17.667558764 podStartE2EDuration="19.451328031s" totalImagesPullingTime="1.783769267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:51:38.450914101 +0000 UTC m=+266.513221127" watchObservedRunningTime="2026-09-11 12:51:38.451328031 +0000 UTC m=+266.513634989"
Sep 11 12:51:41.041590 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:41.041546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:41.042034 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:41.041613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:41.042034 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:41.041726    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:41.042034 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:41.041802    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls podName:b8a891b1-0df9-4a78-9936-82eea74ff958 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:57.04178332 +0000 UTC m=+285.104090279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2c55d" (UID: "b8a891b1-0df9-4a78-9936-82eea74ff958") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:51:41.042267 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:41.042249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8a891b1-0df9-4a78-9936-82eea74ff958-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:51.936741 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:51.936707    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bc49674fd-2hgfz"]
Sep 11 12:51:51.937226 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:51.936954    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz" podUID="284ed70f-dcfc-48d9-aa7b-b323e18955de"
Sep 11 12:51:52.045818 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.045782    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d89946d9d-54mhv"]
Sep 11 12:51:52.050890 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.050865    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.067252 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.067219    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d89946d9d-54mhv"]
Sep 11 12:51:52.125281 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.125238    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8c10a83-cc71-43f8-b38a-476e15357fa8-installation-pull-secrets\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.125281 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.125285    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8c10a83-cc71-43f8-b38a-476e15357fa8-trusted-ca\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.125539 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.125363    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8c10a83-cc71-43f8-b38a-476e15357fa8-registry-tls\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.125539 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.125390    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8c10a83-cc71-43f8-b38a-476e15357fa8-registry-certificates\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.125539 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.125415    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7nw8\" (UniqueName: \"kubernetes.io/projected/f8c10a83-cc71-43f8-b38a-476e15357fa8-kube-api-access-x7nw8\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.125539 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.125447    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8c10a83-cc71-43f8-b38a-476e15357fa8-ca-trust-extracted\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.125539 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.125477    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8c10a83-cc71-43f8-b38a-476e15357fa8-image-registry-private-configuration\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.125539 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.125516    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8c10a83-cc71-43f8-b38a-476e15357fa8-bound-sa-token\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.226846 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.226755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8c10a83-cc71-43f8-b38a-476e15357fa8-installation-pull-secrets\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.226846 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.226796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8c10a83-cc71-43f8-b38a-476e15357fa8-trusted-ca\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.227072 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.226929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8c10a83-cc71-43f8-b38a-476e15357fa8-registry-tls\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.227072 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.226945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8c10a83-cc71-43f8-b38a-476e15357fa8-registry-certificates\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.227072 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.226965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7nw8\" (UniqueName: \"kubernetes.io/projected/f8c10a83-cc71-43f8-b38a-476e15357fa8-kube-api-access-x7nw8\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.227072 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.227000    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8c10a83-cc71-43f8-b38a-476e15357fa8-ca-trust-extracted\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.227072 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.227035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8c10a83-cc71-43f8-b38a-476e15357fa8-image-registry-private-configuration\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.227072 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.227069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8c10a83-cc71-43f8-b38a-476e15357fa8-bound-sa-token\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.227453 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.227430    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8c10a83-cc71-43f8-b38a-476e15357fa8-ca-trust-extracted\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.228016 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.227987    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8c10a83-cc71-43f8-b38a-476e15357fa8-registry-certificates\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.228235 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.228210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8c10a83-cc71-43f8-b38a-476e15357fa8-trusted-ca\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.229495 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.229471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8c10a83-cc71-43f8-b38a-476e15357fa8-image-registry-private-configuration\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.229576 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.229493    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8c10a83-cc71-43f8-b38a-476e15357fa8-installation-pull-secrets\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.229787 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.229771    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8c10a83-cc71-43f8-b38a-476e15357fa8-registry-tls\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.241282 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.241247    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8c10a83-cc71-43f8-b38a-476e15357fa8-bound-sa-token\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.243186 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.243157    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7nw8\" (UniqueName: \"kubernetes.io/projected/f8c10a83-cc71-43f8-b38a-476e15357fa8-kube-api-access-x7nw8\") pod \"image-registry-7d89946d9d-54mhv\" (UID: \"f8c10a83-cc71-43f8-b38a-476e15357fa8\") " pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.367544 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.367513    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wqf2z\""
Sep 11 12:51:52.371769 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.371745    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:52.476739 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.476704    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:51:52.501558 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.501484    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:51:52.528890 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.528856    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bhj7\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-kube-api-access-7bhj7\") pod \"284ed70f-dcfc-48d9-aa7b-b323e18955de\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") "
Sep 11 12:51:52.528967 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.528906    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-trusted-ca\") pod \"284ed70f-dcfc-48d9-aa7b-b323e18955de\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") "
Sep 11 12:51:52.528967 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.528936    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-bound-sa-token\") pod \"284ed70f-dcfc-48d9-aa7b-b323e18955de\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") "
Sep 11 12:51:52.529051 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.528982    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-installation-pull-secrets\") pod \"284ed70f-dcfc-48d9-aa7b-b323e18955de\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") "
Sep 11 12:51:52.529051 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.529013    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-image-registry-private-configuration\") pod \"284ed70f-dcfc-48d9-aa7b-b323e18955de\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") "
Sep 11 12:51:52.529051 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.529042    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-certificates\") pod \"284ed70f-dcfc-48d9-aa7b-b323e18955de\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") "
Sep 11 12:51:52.529178 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.529111    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/284ed70f-dcfc-48d9-aa7b-b323e18955de-ca-trust-extracted\") pod \"284ed70f-dcfc-48d9-aa7b-b323e18955de\" (UID: \"284ed70f-dcfc-48d9-aa7b-b323e18955de\") "
Sep 11 12:51:52.529619 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.529557    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/284ed70f-dcfc-48d9-aa7b-b323e18955de-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "284ed70f-dcfc-48d9-aa7b-b323e18955de" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:51:52.529619 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.529564    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "284ed70f-dcfc-48d9-aa7b-b323e18955de" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:51:52.529619 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.529612    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "284ed70f-dcfc-48d9-aa7b-b323e18955de" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:51:52.531431 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.531401    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "284ed70f-dcfc-48d9-aa7b-b323e18955de" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:51:52.531531 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.531409    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-kube-api-access-7bhj7" (OuterVolumeSpecName: "kube-api-access-7bhj7") pod "284ed70f-dcfc-48d9-aa7b-b323e18955de" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de"). InnerVolumeSpecName "kube-api-access-7bhj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:51:52.531732 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.531715    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "284ed70f-dcfc-48d9-aa7b-b323e18955de" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:51:52.531799 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.531748    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "284ed70f-dcfc-48d9-aa7b-b323e18955de" (UID: "284ed70f-dcfc-48d9-aa7b-b323e18955de"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:51:52.630259 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.630223    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bhj7\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-kube-api-access-7bhj7\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:51:52.630259 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.630251    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-trusted-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:51:52.630259 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.630262    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-bound-sa-token\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:51:52.630259 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.630270    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-installation-pull-secrets\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:51:52.630524 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.630279    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/284ed70f-dcfc-48d9-aa7b-b323e18955de-image-registry-private-configuration\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:51:52.630524 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.630289    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-certificates\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:51:52.630524 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.630299    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/284ed70f-dcfc-48d9-aa7b-b323e18955de-ca-trust-extracted\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:51:52.706348 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:52.706162    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d89946d9d-54mhv"]
Sep 11 12:51:52.708840 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:51:52.708811    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c10a83_cc71_43f8_b38a_476e15357fa8.slice/crio-9bbf9acc9a53e075d610cc01e96547a56dba0e18e5f6b4025b946ba054592a15 WatchSource:0}: Error finding container 9bbf9acc9a53e075d610cc01e96547a56dba0e18e5f6b4025b946ba054592a15: Status 404 returned error can't find the container with id 9bbf9acc9a53e075d610cc01e96547a56dba0e18e5f6b4025b946ba054592a15
Sep 11 12:51:53.480931 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:53.480887    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d89946d9d-54mhv" event={"ID":"f8c10a83-cc71-43f8-b38a-476e15357fa8","Type":"ContainerStarted","Data":"2ac8ac04891972051d4a1148f71690b6bdf2da34f7dad4a40d81f7e95d861ef5"}
Sep 11 12:51:53.480931 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:53.480937    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d89946d9d-54mhv" event={"ID":"f8c10a83-cc71-43f8-b38a-476e15357fa8","Type":"ContainerStarted","Data":"9bbf9acc9a53e075d610cc01e96547a56dba0e18e5f6b4025b946ba054592a15"}
Sep 11 12:51:53.481433 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:53.480903    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bc49674fd-2hgfz"
Sep 11 12:51:53.481433 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:53.481222    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:51:53.523142 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:53.523087    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d89946d9d-54mhv" podStartSLOduration=1.523068906 podStartE2EDuration="1.523068906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:51:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:51:53.522046931 +0000 UTC m=+281.584353890" watchObservedRunningTime="2026-09-11 12:51:53.523068906 +0000 UTC m=+281.585375863"
Sep 11 12:51:53.594709 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:53.594679    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bc49674fd-2hgfz"]
Sep 11 12:51:53.613990 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:53.613964    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bc49674fd-2hgfz"]
Sep 11 12:51:53.637369 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:53.637345    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/284ed70f-dcfc-48d9-aa7b-b323e18955de-registry-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:51:54.110750 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:54.110706    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rt827" podUID="4b04ebea-779c-4b76-9ebb-d66ffa34ee02"
Sep 11 12:51:54.481232 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:54.481156    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="284ed70f-dcfc-48d9-aa7b-b323e18955de" path="/var/lib/kubelet/pods/284ed70f-dcfc-48d9-aa7b-b323e18955de/volumes"
Sep 11 12:51:54.483691 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:54.483669    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rt827"
Sep 11 12:51:54.870267 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:54.870234    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"]
Sep 11 12:51:54.873613 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:54.873596    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"
Sep 11 12:51:54.876446 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:54.876425    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 12:51:54.876795 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:54.876779    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-x7fn9\""
Sep 11 12:51:54.890586 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:54.890557    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"]
Sep 11 12:51:54.947096 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:54.947061    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/85beb836-f149-472f-ae2a-449d9224c542-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-mk5hl\" (UID: \"85beb836-f149-472f-ae2a-449d9224c542\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"
Sep 11 12:51:55.048028 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:55.047979    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/85beb836-f149-472f-ae2a-449d9224c542-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-mk5hl\" (UID: \"85beb836-f149-472f-ae2a-449d9224c542\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"
Sep 11 12:51:55.048190 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:55.048130    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 11 12:51:55.048230 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:51:55.048219    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85beb836-f149-472f-ae2a-449d9224c542-tls-certificates podName:85beb836-f149-472f-ae2a-449d9224c542 nodeName:}" failed. No retries permitted until 2026-09-11 12:51:55.548201925 +0000 UTC m=+283.610508876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/85beb836-f149-472f-ae2a-449d9224c542-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-mk5hl" (UID: "85beb836-f149-472f-ae2a-449d9224c542") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 11 12:51:55.551783 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:55.551744    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/85beb836-f149-472f-ae2a-449d9224c542-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-mk5hl\" (UID: \"85beb836-f149-472f-ae2a-449d9224c542\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"
Sep 11 12:51:55.554245 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:55.554220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/85beb836-f149-472f-ae2a-449d9224c542-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-mk5hl\" (UID: \"85beb836-f149-472f-ae2a-449d9224c542\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"
Sep 11 12:51:55.782354 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:55.782318    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"
Sep 11 12:51:56.060734 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:56.060700    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"]
Sep 11 12:51:56.063547 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:51:56.063508    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85beb836_f149_472f_ae2a_449d9224c542.slice/crio-b75c15c201a0104820fa458132e2448134a61d2b042d034eab5ae263adfb24c3 WatchSource:0}: Error finding container b75c15c201a0104820fa458132e2448134a61d2b042d034eab5ae263adfb24c3: Status 404 returned error can't find the container with id b75c15c201a0104820fa458132e2448134a61d2b042d034eab5ae263adfb24c3
Sep 11 12:51:56.491461 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:56.491426    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl" event={"ID":"85beb836-f149-472f-ae2a-449d9224c542","Type":"ContainerStarted","Data":"b75c15c201a0104820fa458132e2448134a61d2b042d034eab5ae263adfb24c3"}
Sep 11 12:51:57.064992 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.064955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:57.067163 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.067133    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8a891b1-0df9-4a78-9936-82eea74ff958-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2c55d\" (UID: \"b8a891b1-0df9-4a78-9936-82eea74ff958\") " pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:57.165874 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.165835    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:51:57.168435 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.168408    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b04ebea-779c-4b76-9ebb-d66ffa34ee02-metrics-tls\") pod \"dns-default-rt827\" (UID: \"4b04ebea-779c-4b76-9ebb-d66ffa34ee02\") " pod="openshift-dns/dns-default-rt827"
Sep 11 12:51:57.187484 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.187451    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2v8d4\""
Sep 11 12:51:57.195060 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.195026    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rt827"
Sep 11 12:51:57.266996 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.266958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2c55d"
Sep 11 12:51:57.495873 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.495842    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl" event={"ID":"85beb836-f149-472f-ae2a-449d9224c542","Type":"ContainerStarted","Data":"68193274855fe973d6fc49d4eb15bd6b67bd66ec3dae08196d924099b814b131"}
Sep 11 12:51:57.496081 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.496060    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"
Sep 11 12:51:57.500674 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.500632    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl"
Sep 11 12:51:57.525086 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.525035    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mk5hl" podStartSLOduration=2.252571811 podStartE2EDuration="3.525018228s" totalImagesPullingTime="1.272446417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:51:57.524444261 +0000 UTC m=+285.586751219" watchObservedRunningTime="2026-09-11 12:51:57.525018228 +0000 UTC m=+285.587325185"
Sep 11 12:51:57.594771 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.594691    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rt827"]
Sep 11 12:51:57.599269 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:51:57.599223    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b04ebea_779c_4b76_9ebb_d66ffa34ee02.slice/crio-7fcd802ee3c2da311fcc4f8f6b0fbfcc130c11fa202a344967633689f17f0256 WatchSource:0}: Error finding container 7fcd802ee3c2da311fcc4f8f6b0fbfcc130c11fa202a344967633689f17f0256: Status 404 returned error can't find the container with id 7fcd802ee3c2da311fcc4f8f6b0fbfcc130c11fa202a344967633689f17f0256
Sep 11 12:51:57.602697 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:57.602636    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2c55d"]
Sep 11 12:51:57.606312 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:51:57.606288    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a891b1_0df9_4a78_9936_82eea74ff958.slice/crio-880b9a9062fcfa50d02e8d206dc623ace8f87db27d242ac3cb438b838ad925bb WatchSource:0}: Error finding container 880b9a9062fcfa50d02e8d206dc623ace8f87db27d242ac3cb438b838ad925bb: Status 404 returned error can't find the container with id 880b9a9062fcfa50d02e8d206dc623ace8f87db27d242ac3cb438b838ad925bb
Sep 11 12:51:58.502416 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:58.502373    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2c55d" event={"ID":"b8a891b1-0df9-4a78-9936-82eea74ff958","Type":"ContainerStarted","Data":"a8136bfde18099603f77a57b170f2370a556684d1c9de893728df08eaff89877"}
Sep 11 12:51:58.502925 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:58.502472    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2c55d" event={"ID":"b8a891b1-0df9-4a78-9936-82eea74ff958","Type":"ContainerStarted","Data":"1b684fa977ff89e4c0ae6f14dd152d7807b0cd33a2a1b05458a34b9d02a1d4f5"}
Sep 11 12:51:58.502925 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:58.502508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2c55d" event={"ID":"b8a891b1-0df9-4a78-9936-82eea74ff958","Type":"ContainerStarted","Data":"880b9a9062fcfa50d02e8d206dc623ace8f87db27d242ac3cb438b838ad925bb"}
Sep 11 12:51:58.503823 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:51:58.503794    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rt827" event={"ID":"4b04ebea-779c-4b76-9ebb-d66ffa34ee02","Type":"ContainerStarted","Data":"7fcd802ee3c2da311fcc4f8f6b0fbfcc130c11fa202a344967633689f17f0256"}
Sep 11 12:52:00.514542 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:00.514502    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rt827" event={"ID":"4b04ebea-779c-4b76-9ebb-d66ffa34ee02","Type":"ContainerStarted","Data":"51506bb29c37d2f09fd1d1034da88bcb34bc4427658d4c9ca06cc56fd2b04856"}
Sep 11 12:52:00.514999 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:00.514570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rt827" event={"ID":"4b04ebea-779c-4b76-9ebb-d66ffa34ee02","Type":"ContainerStarted","Data":"b9023a6cbb03b1592538e4e3f31ee7402b61d6fcc884ff951de455d47fba7812"}
Sep 11 12:52:00.514999 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:00.514609    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rt827"
Sep 11 12:52:00.516391 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:00.516370    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2c55d" event={"ID":"b8a891b1-0df9-4a78-9936-82eea74ff958","Type":"ContainerStarted","Data":"e21a4e578ef843881e003703c2c7787edd0f330a2ff282afeadb99f3b1c2b37a"}
Sep 11 12:52:00.534074 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:00.534029    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rt827" podStartSLOduration=251.357863012 podStartE2EDuration="4m13.534013091s" totalImagesPullingTime="2.176150079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:47:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:52:00.533013197 +0000 UTC m=+288.595320154" watchObservedRunningTime="2026-09-11 12:52:00.534013091 +0000 UTC m=+288.596320048"
Sep 11 12:52:00.555231 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:00.555181    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2c55d" podStartSLOduration=33.519597286 podStartE2EDuration="35.555163658s" totalImagesPullingTime="2.035566372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:51:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:52:00.554677528 +0000 UTC m=+288.616984485" watchObservedRunningTime="2026-09-11 12:52:00.555163658 +0000 UTC m=+288.617470616"
Sep 11 12:52:02.301846 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.301810    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"]
Sep 11 12:52:02.304618 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.304601    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.307653 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.307618    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 12:52:02.308204 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.308180    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 12:52:02.309123 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.309106    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 12:52:02.309224 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.309132    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 12:52:02.309224 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.309183    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 12:52:02.309224 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.309214    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zkx4l\""
Sep 11 12:52:02.318541 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.318521    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"]
Sep 11 12:52:02.356889 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.356853    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-j7g44"]
Sep 11 12:52:02.359675 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.359634    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.362730 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.362701    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8ttnf\""
Sep 11 12:52:02.362832 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.362761    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 12:52:02.363173 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.363155    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 12:52:02.363316 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.363299    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 12:52:02.405207 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.405207 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb35d9cb-a077-48dd-bff8-8416801e46b0-sys\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.405419 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405226    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69rrl\" (UniqueName: \"kubernetes.io/projected/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-kube-api-access-69rrl\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.405419 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405261    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-wtmp\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.405419 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405281    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-tls\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.405419 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405302    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.405419 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405319    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-accelerators-collector-config\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.405419 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.405419 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405361    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eb35d9cb-a077-48dd-bff8-8416801e46b0-root\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.405419 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405376    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-textfile\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.405419 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405392    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9tgl\" (UniqueName: \"kubernetes.io/projected/eb35d9cb-a077-48dd-bff8-8416801e46b0-kube-api-access-d9tgl\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.405735 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.405735 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.405498    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb35d9cb-a077-48dd-bff8-8416801e46b0-metrics-client-ca\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506149 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506112    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.506149 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506149    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-accelerators-collector-config\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506425 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.506425 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506200    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eb35d9cb-a077-48dd-bff8-8416801e46b0-root\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506425 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506225    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-textfile\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506425 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506249    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9tgl\" (UniqueName: \"kubernetes.io/projected/eb35d9cb-a077-48dd-bff8-8416801e46b0-kube-api-access-d9tgl\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506425 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506281    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eb35d9cb-a077-48dd-bff8-8416801e46b0-root\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506425 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.506425 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:52:02.506364    2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 12:52:02.506425 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506399    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb35d9cb-a077-48dd-bff8-8416801e46b0-metrics-client-ca\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506425 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:52:02.506423    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-openshift-state-metrics-tls podName:68694e75-9dc4-41a8-9a0d-6b8b07d638c9 nodeName:}" failed. No retries permitted until 2026-09-11 12:52:03.00640408 +0000 UTC m=+291.068711030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-lqtlx" (UID: "68694e75-9dc4-41a8-9a0d-6b8b07d638c9") : secret "openshift-state-metrics-tls" not found
Sep 11 12:52:02.506919 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506919 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506496    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb35d9cb-a077-48dd-bff8-8416801e46b0-sys\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506919 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506525    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69rrl\" (UniqueName: \"kubernetes.io/projected/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-kube-api-access-69rrl\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.506919 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506548    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-wtmp\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506919 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-tls\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506919 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506720    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-textfile\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506919 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506789    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb35d9cb-a077-48dd-bff8-8416801e46b0-sys\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.506919 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-wtmp\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.507250 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.506956    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-accelerators-collector-config\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.507250 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.507062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.507387 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.507364    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb35d9cb-a077-48dd-bff8-8416801e46b0-metrics-client-ca\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.508871 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.508852    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-tls\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.508871 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.508859    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb35d9cb-a077-48dd-bff8-8416801e46b0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.509041 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.508874    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.519895 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.519858    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9tgl\" (UniqueName: \"kubernetes.io/projected/eb35d9cb-a077-48dd-bff8-8416801e46b0-kube-api-access-d9tgl\") pod \"node-exporter-j7g44\" (UID: \"eb35d9cb-a077-48dd-bff8-8416801e46b0\") " pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.521942 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.521919    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69rrl\" (UniqueName: \"kubernetes.io/projected/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-kube-api-access-69rrl\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:02.668228 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:02.668187    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j7g44"
Sep 11 12:52:02.678413 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:52:02.678373    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb35d9cb_a077_48dd_bff8_8416801e46b0.slice/crio-680e5036d7e3df8def32b8d05eadcee8c2767b1c7a900add538049e5484f9dec WatchSource:0}: Error finding container 680e5036d7e3df8def32b8d05eadcee8c2767b1c7a900add538049e5484f9dec: Status 404 returned error can't find the container with id 680e5036d7e3df8def32b8d05eadcee8c2767b1c7a900add538049e5484f9dec
Sep 11 12:52:03.011219 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:03.011123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:03.013565 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:03.013534    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68694e75-9dc4-41a8-9a0d-6b8b07d638c9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-lqtlx\" (UID: \"68694e75-9dc4-41a8-9a0d-6b8b07d638c9\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:03.214255 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:03.214213    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"
Sep 11 12:52:03.498259 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:03.498218    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx"]
Sep 11 12:52:03.528952 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:03.528865    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j7g44" event={"ID":"eb35d9cb-a077-48dd-bff8-8416801e46b0","Type":"ContainerStarted","Data":"680e5036d7e3df8def32b8d05eadcee8c2767b1c7a900add538049e5484f9dec"}
Sep 11 12:52:03.530533 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:52:03.530495    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68694e75_9dc4_41a8_9a0d_6b8b07d638c9.slice/crio-4443579a0a74192feb2fa12f365b96785fc761e3c3e96617d578a850cd55e92c WatchSource:0}: Error finding container 4443579a0a74192feb2fa12f365b96785fc761e3c3e96617d578a850cd55e92c: Status 404 returned error can't find the container with id 4443579a0a74192feb2fa12f365b96785fc761e3c3e96617d578a850cd55e92c
Sep 11 12:52:04.535616 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:04.535577    2564 generic.go:356] "Generic (PLEG): container finished" podID="eb35d9cb-a077-48dd-bff8-8416801e46b0" containerID="a24b6efbb21fad56daf916dc68202bc76341755d09e2fcb93e07575929bb53e2" exitCode=0
Sep 11 12:52:04.536096 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:04.535682    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j7g44" event={"ID":"eb35d9cb-a077-48dd-bff8-8416801e46b0","Type":"ContainerDied","Data":"a24b6efbb21fad56daf916dc68202bc76341755d09e2fcb93e07575929bb53e2"}
Sep 11 12:52:04.538007 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:04.537975    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx" event={"ID":"68694e75-9dc4-41a8-9a0d-6b8b07d638c9","Type":"ContainerStarted","Data":"de7ba1d4de0c7267e2c65b1e139b3c473dea0d68fda3cd183668db67a1e229cf"}
Sep 11 12:52:04.538126 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:04.538036    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx" event={"ID":"68694e75-9dc4-41a8-9a0d-6b8b07d638c9","Type":"ContainerStarted","Data":"7f3a6838b6d190a904bd6037d71d4f31dfcd6dfe8562a3bbb71a7a1cf510abfb"}
Sep 11 12:52:04.538126 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:04.538057    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx" event={"ID":"68694e75-9dc4-41a8-9a0d-6b8b07d638c9","Type":"ContainerStarted","Data":"4443579a0a74192feb2fa12f365b96785fc761e3c3e96617d578a850cd55e92c"}
Sep 11 12:52:05.308151 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.308117    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"]
Sep 11 12:52:05.310711 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.310692    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.313579 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.313548    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7v8xp\""
Sep 11 12:52:05.313710 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.313603    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-emp10nn126l1n\""
Sep 11 12:52:05.313710 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.313623    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 12:52:05.313710 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.313624    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 12:52:05.313710 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.313693    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 12:52:05.313710 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.313701    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 12:52:05.313943 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.313769    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 12:52:05.325763 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.325735    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"]
Sep 11 12:52:05.432809 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.432766    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-metrics-client-ca\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.432966 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.432816    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.432966 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.432845    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.432966 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.432930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-grpc-tls\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.433095 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.432972    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.433095 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.433043    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.433095 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.433077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-tls\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.433095 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.433093    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6l9g\" (UniqueName: \"kubernetes.io/projected/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-kube-api-access-s6l9g\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.534217 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.534177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-metrics-client-ca\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.534402 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.534227    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.534402 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.534349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.534515 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.534451    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-grpc-tls\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.534515 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.534494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.534605 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.534546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.534685 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.534600    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-tls\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.534685 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.534627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6l9g\" (UniqueName: \"kubernetes.io/projected/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-kube-api-access-s6l9g\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.535001 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.534969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-metrics-client-ca\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.536994 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.536970    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.537393 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.537272    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-tls\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.537393 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.537342    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.537393 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.537347    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.537543 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.537460    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.537808 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.537789    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-secret-grpc-tls\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.542949 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.542929    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6l9g\" (UniqueName: \"kubernetes.io/projected/6f351282-6bb3-49b0-98a7-3629e2d4a3aa-kube-api-access-s6l9g\") pod \"thanos-querier-5b9f49dc4d-zdr8h\" (UID: \"6f351282-6bb3-49b0-98a7-3629e2d4a3aa\") " pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.544136 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.544105    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j7g44" event={"ID":"eb35d9cb-a077-48dd-bff8-8416801e46b0","Type":"ContainerStarted","Data":"4ace7b4ff1367a6cc4eda5f71b652d19c3962efd61d8d5d982897c3b66722d3f"}
Sep 11 12:52:05.544235 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.544172    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j7g44" event={"ID":"eb35d9cb-a077-48dd-bff8-8416801e46b0","Type":"ContainerStarted","Data":"094b96ffe3b0f2c11eb0f882a4c154eab6ab348d235e5e653a8bfed31af98fe1"}
Sep 11 12:52:05.546469 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.546433    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx" event={"ID":"68694e75-9dc4-41a8-9a0d-6b8b07d638c9","Type":"ContainerStarted","Data":"be600d06473a25c3051ca94f2a834d8a550360def5a0519ce17d5344210eab95"}
Sep 11 12:52:05.563442 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.563349    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-j7g44" podStartSLOduration=2.613013527 podStartE2EDuration="3.563333199s" totalImagesPullingTime="875.280138ms" totalInitContainerRuntime="75.039534ms" isStatefulPod=true podCreationTimestamp="2026-09-11 12:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:52:05.562280387 +0000 UTC m=+293.624587339" watchObservedRunningTime="2026-09-11 12:52:05.563333199 +0000 UTC m=+293.625640154"
Sep 11 12:52:05.582556 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.582504    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-lqtlx" podStartSLOduration=2.689273546 podStartE2EDuration="3.582488548s" totalImagesPullingTime="893.215002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:52:05.582392967 +0000 UTC m=+293.644699919" watchObservedRunningTime="2026-09-11 12:52:05.582488548 +0000 UTC m=+293.644795506"
Sep 11 12:52:05.619918 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.619879    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:05.897144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:05.897115    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"]
Sep 11 12:52:05.898839 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:52:05.898808    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f351282_6bb3_49b0_98a7_3629e2d4a3aa.slice/crio-860d6a2d514411598443fd0275dc9cb223f585022e428ba31312dac20fe6be9c WatchSource:0}: Error finding container 860d6a2d514411598443fd0275dc9cb223f585022e428ba31312dac20fe6be9c: Status 404 returned error can't find the container with id 860d6a2d514411598443fd0275dc9cb223f585022e428ba31312dac20fe6be9c
Sep 11 12:52:06.552156 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:06.552110    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h" event={"ID":"6f351282-6bb3-49b0-98a7-3629e2d4a3aa","Type":"ContainerStarted","Data":"860d6a2d514411598443fd0275dc9cb223f585022e428ba31312dac20fe6be9c"}
Sep 11 12:52:07.477527 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:52:07.477492    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4f7x7" podUID="7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028"
Sep 11 12:52:08.563339 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:08.563298    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h" event={"ID":"6f351282-6bb3-49b0-98a7-3629e2d4a3aa","Type":"ContainerStarted","Data":"820689d4684f5199498c5c8e9defc96a50e06b0b84b10da7d1d54d91787f1b0b"}
Sep 11 12:52:08.563782 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:08.563390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h" event={"ID":"6f351282-6bb3-49b0-98a7-3629e2d4a3aa","Type":"ContainerStarted","Data":"99f5f237f1a103e50f0fccfc99cdf5c98a8ca478b003adb72f49b71374077035"}
Sep 11 12:52:08.563782 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:08.563433    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h" event={"ID":"6f351282-6bb3-49b0-98a7-3629e2d4a3aa","Type":"ContainerStarted","Data":"bab3f7c21a840bcc5294e4fa685244ad97281530428da400bb90ef500fb40031"}
Sep 11 12:52:09.570670 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:09.570625    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h" event={"ID":"6f351282-6bb3-49b0-98a7-3629e2d4a3aa","Type":"ContainerStarted","Data":"5240cc4d59edaee77e4e7467459338890e797c65118569c8007ad4f1f403e007"}
Sep 11 12:52:09.571045 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:09.570704    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h" event={"ID":"6f351282-6bb3-49b0-98a7-3629e2d4a3aa","Type":"ContainerStarted","Data":"b11053c05256833705f7576a02a902de7505f9bf5ecb1aeff7ccd81baafbf23b"}
Sep 11 12:52:09.571045 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:09.570731    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h" event={"ID":"6f351282-6bb3-49b0-98a7-3629e2d4a3aa","Type":"ContainerStarted","Data":"77d51a87f666dd32d98d1014c43358f5bfa4f73bc3adb5265ca434d3e4f0ef70"}
Sep 11 12:52:09.571045 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:09.570798    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:09.597222 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:09.597172    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h" podStartSLOduration=1.77369984 podStartE2EDuration="4.597155229s" totalImagesPullingTime="2.823455389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:52:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:52:09.595033092 +0000 UTC m=+297.657340050" watchObservedRunningTime="2026-09-11 12:52:09.597155229 +0000 UTC m=+297.659462188"
Sep 11 12:52:10.522810 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:10.522775    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rt827"
Sep 11 12:52:12.391478 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:12.391451    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 12:52:12.392955 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:12.392932    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 12:52:12.395716 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:12.395696    2564 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 12:52:14.488690 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:14.488631    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d89946d9d-54mhv"
Sep 11 12:52:15.580333 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:15.580301    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b9f49dc4d-zdr8h"
Sep 11 12:52:22.479892 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:22.479854    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:52:29.648503 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:29.648413    2564 generic.go:356] "Generic (PLEG): container finished" podID="cdad3f72-dc2b-423e-8cd5-cbb2445667fb" containerID="d7d3546e5b66daeae73e71eab75880e679c56a4bfee1c421229706c2eb669d69" exitCode=0
Sep 11 12:52:29.648503 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:29.648486    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qzskx" event={"ID":"cdad3f72-dc2b-423e-8cd5-cbb2445667fb","Type":"ContainerDied","Data":"d7d3546e5b66daeae73e71eab75880e679c56a4bfee1c421229706c2eb669d69"}
Sep 11 12:52:29.648945 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:29.648823    2564 scope.go:117] "RemoveContainer" containerID="d7d3546e5b66daeae73e71eab75880e679c56a4bfee1c421229706c2eb669d69"
Sep 11 12:52:30.541122 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:30.541067    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nlctv_53e588b7-21cd-4c00-bfb7-68002b45b7a0/serve-healthcheck-canary/0.log"
Sep 11 12:52:30.654837 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:30.654799    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qzskx" event={"ID":"cdad3f72-dc2b-423e-8cd5-cbb2445667fb","Type":"ContainerStarted","Data":"465b1d28b9f54722784a3ec9debcdc70cd8005783ed1adf8bed91e090f757840"}
Sep 11 12:52:30.739860 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:30.739813    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-86499c89c4-2br76_c4485948-2373-425a-8295-5b95217023b2/cluster-samples-operator/0.log"
Sep 11 12:52:30.938949 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:30.938920    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-86499c89c4-2br76_c4485948-2373-425a-8295-5b95217023b2/cluster-samples-operator-watch/0.log"
Sep 11 12:52:49.731043 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:49.731007    2564 generic.go:356] "Generic (PLEG): container finished" podID="c859ee38-4f38-46b2-8b8b-ae6345444bc6" containerID="c668a2383ba8775fc2783d6a3c072f3a20109f904c3c9c0c4842728f3664ffeb" exitCode=0
Sep 11 12:52:49.731446 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:49.731083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w" event={"ID":"c859ee38-4f38-46b2-8b8b-ae6345444bc6","Type":"ContainerDied","Data":"c668a2383ba8775fc2783d6a3c072f3a20109f904c3c9c0c4842728f3664ffeb"}
Sep 11 12:52:49.731446 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:49.731438    2564 scope.go:117] "RemoveContainer" containerID="c668a2383ba8775fc2783d6a3c072f3a20109f904c3c9c0c4842728f3664ffeb"
Sep 11 12:52:50.736696 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:50.736660    2564 generic.go:356] "Generic (PLEG): container finished" podID="8583207f-fafa-425c-8d07-cdf8a61083ab" containerID="6ac1f41009ba0d07bdac30d57c85ac617ec904a7cacf472f6390d58301533737" exitCode=0
Sep 11 12:52:50.737148 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:50.736713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z" event={"ID":"8583207f-fafa-425c-8d07-cdf8a61083ab","Type":"ContainerDied","Data":"6ac1f41009ba0d07bdac30d57c85ac617ec904a7cacf472f6390d58301533737"}
Sep 11 12:52:50.737148 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:50.737088    2564 scope.go:117] "RemoveContainer" containerID="6ac1f41009ba0d07bdac30d57c85ac617ec904a7cacf472f6390d58301533737"
Sep 11 12:52:50.738889 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:50.738866    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-l2k5w" event={"ID":"c859ee38-4f38-46b2-8b8b-ae6345444bc6","Type":"ContainerStarted","Data":"df68cd96788aa726bf87ef53853043e23213796f5e032b40c98ef62882a10abb"}
Sep 11 12:52:51.744106 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:52:51.744077    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-wsg8z" event={"ID":"8583207f-fafa-425c-8d07-cdf8a61083ab","Type":"ContainerStarted","Data":"e3417c3384464441b16719f183b2524e60308605fd68d4268b2c0d8eb5ce5f99"}
Sep 11 12:53:25.309127 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:53:25.309036    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:53:25.311329 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:53:25.311307    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028-metrics-certs\") pod \"network-metrics-daemon-4f7x7\" (UID: \"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028\") " pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:53:25.483408 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:53:25.483373    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-86zcf\""
Sep 11 12:53:25.491442 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:53:25.491411    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4f7x7"
Sep 11 12:53:25.767447 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:53:25.767411    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4f7x7"]
Sep 11 12:53:25.770713 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:53:25.770682    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5e9cd8_9227_4dc0_84c8_1ee84ee9e028.slice/crio-e04d93138c0255542d42dbbf874e2dde798529ef37f799400034bb9c1744b3c9 WatchSource:0}: Error finding container e04d93138c0255542d42dbbf874e2dde798529ef37f799400034bb9c1744b3c9: Status 404 returned error can't find the container with id e04d93138c0255542d42dbbf874e2dde798529ef37f799400034bb9c1744b3c9
Sep 11 12:53:25.773251 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:53:25.773235    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:53:25.876312 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:53:25.876273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4f7x7" event={"ID":"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028","Type":"ContainerStarted","Data":"e04d93138c0255542d42dbbf874e2dde798529ef37f799400034bb9c1744b3c9"}
Sep 11 12:53:26.882977 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:53:26.882912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4f7x7" event={"ID":"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028","Type":"ContainerStarted","Data":"5baf988729815be517170ef20411fa5c2db4367559d750ac1d8a2d85a1969ab4"}
Sep 11 12:53:27.888698 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:53:27.888660    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4f7x7" event={"ID":"7c5e9cd8-9227-4dc0-84c8-1ee84ee9e028","Type":"ContainerStarted","Data":"8a91243518fef838db536aeb3210ad564a8c281b35e23e1ed38d495ff61b7d15"}
Sep 11 12:53:27.906555 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:53:27.906505    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4f7x7" podStartSLOduration=373.911391243 podStartE2EDuration="6m14.90648757s" totalImagesPullingTime="995.096327ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:53:27.904684572 +0000 UTC m=+375.966991526" watchObservedRunningTime="2026-09-11 12:53:27.90648757 +0000 UTC m=+375.968794528"
Sep 11 12:54:53.489624 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.489586    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"]
Sep 11 12:54:53.492739 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.492718    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"
Sep 11 12:54:53.495415 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.495382    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 11 12:54:53.495567 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.495421    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 11 12:54:53.495567 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.495442    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-74wl2\""
Sep 11 12:54:53.508054 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.508028    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"]
Sep 11 12:54:53.582065 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.582019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmxzh\" (UniqueName: \"kubernetes.io/projected/fe0ef325-d838-447c-a71d-cea86090c497-kube-api-access-bmxzh\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4m6j\" (UID: \"fe0ef325-d838-447c-a71d-cea86090c497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"
Sep 11 12:54:53.582249 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.582075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe0ef325-d838-447c-a71d-cea86090c497-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4m6j\" (UID: \"fe0ef325-d838-447c-a71d-cea86090c497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"
Sep 11 12:54:53.682879 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.682834    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmxzh\" (UniqueName: \"kubernetes.io/projected/fe0ef325-d838-447c-a71d-cea86090c497-kube-api-access-bmxzh\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4m6j\" (UID: \"fe0ef325-d838-447c-a71d-cea86090c497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"
Sep 11 12:54:53.683031 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.682900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe0ef325-d838-447c-a71d-cea86090c497-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4m6j\" (UID: \"fe0ef325-d838-447c-a71d-cea86090c497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"
Sep 11 12:54:53.683309 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.683288    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe0ef325-d838-447c-a71d-cea86090c497-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4m6j\" (UID: \"fe0ef325-d838-447c-a71d-cea86090c497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"
Sep 11 12:54:53.691938 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.691899    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmxzh\" (UniqueName: \"kubernetes.io/projected/fe0ef325-d838-447c-a71d-cea86090c497-kube-api-access-bmxzh\") pod \"cert-manager-operator-controller-manager-55c444bf86-v4m6j\" (UID: \"fe0ef325-d838-447c-a71d-cea86090c497\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"
Sep 11 12:54:53.802142 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:53.802055    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"
Sep 11 12:54:54.077512 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:54.077487    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j"]
Sep 11 12:54:54.080548 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:54:54.080522    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe0ef325_d838_447c_a71d_cea86090c497.slice/crio-004a6c48316bdb0b1d1e6d61382c7e18d5a1fc072598e5983c27c948c0086e5a WatchSource:0}: Error finding container 004a6c48316bdb0b1d1e6d61382c7e18d5a1fc072598e5983c27c948c0086e5a: Status 404 returned error can't find the container with id 004a6c48316bdb0b1d1e6d61382c7e18d5a1fc072598e5983c27c948c0086e5a
Sep 11 12:54:54.227716 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:54:54.227680    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j" event={"ID":"fe0ef325-d838-447c-a71d-cea86090c497","Type":"ContainerStarted","Data":"004a6c48316bdb0b1d1e6d61382c7e18d5a1fc072598e5983c27c948c0086e5a"}
Sep 11 12:55:04.273540 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.273502    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j" event={"ID":"fe0ef325-d838-447c-a71d-cea86090c497","Type":"ContainerStarted","Data":"63a3ab230d2f9496df0efade1168f57a756aa00e20681e1146361e0a78c20d31"}
Sep 11 12:55:04.294981 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.294920    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v4m6j" podStartSLOduration=1.702041514 podStartE2EDuration="11.294899341s" totalImagesPullingTime="9.592857827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:55:04.292256701 +0000 UTC m=+472.354563676" watchObservedRunningTime="2026-09-11 12:55:04.294899341 +0000 UTC m=+472.357206299"
Sep 11 12:55:04.580180 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.580150    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qv2tl"]
Sep 11 12:55:04.583502 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.583481    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl"
Sep 11 12:55:04.586030 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.586002    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-npxqc\""
Sep 11 12:55:04.586163 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.586009    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 12:55:04.586163 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.586004    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 12:55:04.592340 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.592307    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qv2tl"]
Sep 11 12:55:04.674996 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.674960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/281b4830-4acb-4fd2-91fc-cdc8f38a07d1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qv2tl\" (UID: \"281b4830-4acb-4fd2-91fc-cdc8f38a07d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl"
Sep 11 12:55:04.674996 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.674998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq68t\" (UniqueName: \"kubernetes.io/projected/281b4830-4acb-4fd2-91fc-cdc8f38a07d1-kube-api-access-lq68t\") pod \"cert-manager-webhook-6486ffd997-qv2tl\" (UID: \"281b4830-4acb-4fd2-91fc-cdc8f38a07d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl"
Sep 11 12:55:04.775991 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.775950    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/281b4830-4acb-4fd2-91fc-cdc8f38a07d1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qv2tl\" (UID: \"281b4830-4acb-4fd2-91fc-cdc8f38a07d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl"
Sep 11 12:55:04.775991 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.775992    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq68t\" (UniqueName: \"kubernetes.io/projected/281b4830-4acb-4fd2-91fc-cdc8f38a07d1-kube-api-access-lq68t\") pod \"cert-manager-webhook-6486ffd997-qv2tl\" (UID: \"281b4830-4acb-4fd2-91fc-cdc8f38a07d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl"
Sep 11 12:55:04.799612 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.799571    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/281b4830-4acb-4fd2-91fc-cdc8f38a07d1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qv2tl\" (UID: \"281b4830-4acb-4fd2-91fc-cdc8f38a07d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl"
Sep 11 12:55:04.799774 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.799746    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq68t\" (UniqueName: \"kubernetes.io/projected/281b4830-4acb-4fd2-91fc-cdc8f38a07d1-kube-api-access-lq68t\") pod \"cert-manager-webhook-6486ffd997-qv2tl\" (UID: \"281b4830-4acb-4fd2-91fc-cdc8f38a07d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl"
Sep 11 12:55:04.893869 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:04.893769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl"
Sep 11 12:55:05.173553 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:05.173529    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qv2tl"]
Sep 11 12:55:05.175111 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:55:05.175087    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod281b4830_4acb_4fd2_91fc_cdc8f38a07d1.slice/crio-fadfeae5fd741f55eb2d13fdd23441a4c3b8f9af4786bf64112cc0532b9e8b16 WatchSource:0}: Error finding container fadfeae5fd741f55eb2d13fdd23441a4c3b8f9af4786bf64112cc0532b9e8b16: Status 404 returned error can't find the container with id fadfeae5fd741f55eb2d13fdd23441a4c3b8f9af4786bf64112cc0532b9e8b16
Sep 11 12:55:05.278801 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:05.278767    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl" event={"ID":"281b4830-4acb-4fd2-91fc-cdc8f38a07d1","Type":"ContainerStarted","Data":"fadfeae5fd741f55eb2d13fdd23441a4c3b8f9af4786bf64112cc0532b9e8b16"}
Sep 11 12:55:08.295126 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:08.295026    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl" event={"ID":"281b4830-4acb-4fd2-91fc-cdc8f38a07d1","Type":"ContainerStarted","Data":"8b74291e7ba613c4d87df5369bd0aca72df0cd9c07248af7eb0be18f40dacafa"}
Sep 11 12:55:08.295126 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:08.295100    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl"
Sep 11 12:55:08.311408 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:08.311361    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl" podStartSLOduration=1.520257141 podStartE2EDuration="4.311345261s" totalImagesPullingTime="2.79108812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:55:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:55:08.310733725 +0000 UTC m=+476.373040682" watchObservedRunningTime="2026-09-11 12:55:08.311345261 +0000 UTC m=+476.373652218"
Sep 11 12:55:14.301853 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:14.301820    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qv2tl"
Sep 11 12:55:24.978223 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:24.978169    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hz9cb"]
Sep 11 12:55:24.984360 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:24.984338    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hz9cb"
Sep 11 12:55:24.986731 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:24.986711    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-tpbkl\""
Sep 11 12:55:24.990446 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:24.990419    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hz9cb"]
Sep 11 12:55:25.029341 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:25.029301    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnn4p\" (UniqueName: \"kubernetes.io/projected/7d48b47f-f088-40fc-95fe-6c2d7811e3a7-kube-api-access-bnn4p\") pod \"cert-manager-55b4d954d5-hz9cb\" (UID: \"7d48b47f-f088-40fc-95fe-6c2d7811e3a7\") " pod="cert-manager/cert-manager-55b4d954d5-hz9cb"
Sep 11 12:55:25.029526 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:25.029376    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d48b47f-f088-40fc-95fe-6c2d7811e3a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-hz9cb\" (UID: \"7d48b47f-f088-40fc-95fe-6c2d7811e3a7\") " pod="cert-manager/cert-manager-55b4d954d5-hz9cb"
Sep 11 12:55:25.130339 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:25.130304    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d48b47f-f088-40fc-95fe-6c2d7811e3a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-hz9cb\" (UID: \"7d48b47f-f088-40fc-95fe-6c2d7811e3a7\") " pod="cert-manager/cert-manager-55b4d954d5-hz9cb"
Sep 11 12:55:25.130542 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:25.130355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnn4p\" (UniqueName: \"kubernetes.io/projected/7d48b47f-f088-40fc-95fe-6c2d7811e3a7-kube-api-access-bnn4p\") pod \"cert-manager-55b4d954d5-hz9cb\" (UID: \"7d48b47f-f088-40fc-95fe-6c2d7811e3a7\") " pod="cert-manager/cert-manager-55b4d954d5-hz9cb"
Sep 11 12:55:25.142045 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:25.142008    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d48b47f-f088-40fc-95fe-6c2d7811e3a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-hz9cb\" (UID: \"7d48b47f-f088-40fc-95fe-6c2d7811e3a7\") " pod="cert-manager/cert-manager-55b4d954d5-hz9cb"
Sep 11 12:55:25.142185 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:25.142084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnn4p\" (UniqueName: \"kubernetes.io/projected/7d48b47f-f088-40fc-95fe-6c2d7811e3a7-kube-api-access-bnn4p\") pod \"cert-manager-55b4d954d5-hz9cb\" (UID: \"7d48b47f-f088-40fc-95fe-6c2d7811e3a7\") " pod="cert-manager/cert-manager-55b4d954d5-hz9cb"
Sep 11 12:55:25.295157 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:25.295051    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hz9cb"
Sep 11 12:55:25.565861 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:25.565771    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hz9cb"]
Sep 11 12:55:25.569481 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:55:25.569448    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d48b47f_f088_40fc_95fe_6c2d7811e3a7.slice/crio-c345713205cd5069a5360217911c08d3a7fe601520947c375c5c8a8cf68a6e78 WatchSource:0}: Error finding container c345713205cd5069a5360217911c08d3a7fe601520947c375c5c8a8cf68a6e78: Status 404 returned error can't find the container with id c345713205cd5069a5360217911c08d3a7fe601520947c375c5c8a8cf68a6e78
Sep 11 12:55:26.378001 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:26.377958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hz9cb" event={"ID":"7d48b47f-f088-40fc-95fe-6c2d7811e3a7","Type":"ContainerStarted","Data":"39b74e65b0b57f650bfaa83dcc8a4a3203d84724b40a8585d278bd4403130b40"}
Sep 11 12:55:26.378387 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:26.378013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hz9cb" event={"ID":"7d48b47f-f088-40fc-95fe-6c2d7811e3a7","Type":"ContainerStarted","Data":"c345713205cd5069a5360217911c08d3a7fe601520947c375c5c8a8cf68a6e78"}
Sep 11 12:55:26.394411 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:26.394352    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-hz9cb" podStartSLOduration=2.394334136 podStartE2EDuration="2.394334136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:55:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:55:26.393540289 +0000 UTC m=+494.455847247" watchObservedRunningTime="2026-09-11 12:55:26.394334136 +0000 UTC m=+494.456641093"
Sep 11 12:55:41.991063 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:41.991028    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"]
Sep 11 12:55:41.993474 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:41.993449    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:41.996024 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:41.995998    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 12:55:41.996177 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:41.996141    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 12:55:41.996247 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:41.996141    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 12:55:41.996247 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:41.996203    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5m6lg\""
Sep 11 12:55:41.997187 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:41.997170    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 12:55:42.012844 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.012814    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"]
Sep 11 12:55:42.046862 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.046822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/814827b4-7a34-4e89-a771-d9cf1df08c6c-tmp\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.046862 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.046867    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/814827b4-7a34-4e89-a771-d9cf1df08c6c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.047087 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.046897    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l9df\" (UniqueName: \"kubernetes.io/projected/814827b4-7a34-4e89-a771-d9cf1df08c6c-kube-api-access-4l9df\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.047087 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.046935    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/814827b4-7a34-4e89-a771-d9cf1df08c6c-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.047087 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.046958    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/814827b4-7a34-4e89-a771-d9cf1df08c6c-manifests\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.147385 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.147336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/814827b4-7a34-4e89-a771-d9cf1df08c6c-manifests\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.147588 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.147401    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/814827b4-7a34-4e89-a771-d9cf1df08c6c-tmp\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.147588 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.147441    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/814827b4-7a34-4e89-a771-d9cf1df08c6c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.147588 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.147475    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l9df\" (UniqueName: \"kubernetes.io/projected/814827b4-7a34-4e89-a771-d9cf1df08c6c-kube-api-access-4l9df\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.147588 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.147529    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/814827b4-7a34-4e89-a771-d9cf1df08c6c-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.147873 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.147836    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/814827b4-7a34-4e89-a771-d9cf1df08c6c-manifests\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.149876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.149841    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/814827b4-7a34-4e89-a771-d9cf1df08c6c-tmp\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.150109 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.150085    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/814827b4-7a34-4e89-a771-d9cf1df08c6c-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.150177 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.150096    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/814827b4-7a34-4e89-a771-d9cf1df08c6c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.156478 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.156439    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l9df\" (UniqueName: \"kubernetes.io/projected/814827b4-7a34-4e89-a771-d9cf1df08c6c-kube-api-access-4l9df\") pod \"opendatahub-operator-controller-manager-7cbb4999dc-gw4wv\" (UID: \"814827b4-7a34-4e89-a771-d9cf1df08c6c\") " pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.304742 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.304612    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:42.581838 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:42.581801    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"]
Sep 11 12:55:42.585190 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:55:42.585153    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod814827b4_7a34_4e89_a771_d9cf1df08c6c.slice/crio-e7b1fa308d450fe8dc917d3cf8095aa0d3bddc916d7e94f9af939588d8f6a6a7 WatchSource:0}: Error finding container e7b1fa308d450fe8dc917d3cf8095aa0d3bddc916d7e94f9af939588d8f6a6a7: Status 404 returned error can't find the container with id e7b1fa308d450fe8dc917d3cf8095aa0d3bddc916d7e94f9af939588d8f6a6a7
Sep 11 12:55:43.460587 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:43.460551    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv" event={"ID":"814827b4-7a34-4e89-a771-d9cf1df08c6c","Type":"ContainerStarted","Data":"e7b1fa308d450fe8dc917d3cf8095aa0d3bddc916d7e94f9af939588d8f6a6a7"}
Sep 11 12:55:46.483401 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:46.483363    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv" event={"ID":"814827b4-7a34-4e89-a771-d9cf1df08c6c","Type":"ContainerStarted","Data":"26d50f36774d1424be69e4ff5e46616135f7c06be46f92fac89aba51cd735ec3"}
Sep 11 12:55:47.485725 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:47.485676    2564 generic.go:356] "Generic (PLEG): container finished" podID="814827b4-7a34-4e89-a771-d9cf1df08c6c" containerID="26d50f36774d1424be69e4ff5e46616135f7c06be46f92fac89aba51cd735ec3" exitCode=0
Sep 11 12:55:47.486127 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:47.485756    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv" event={"ID":"814827b4-7a34-4e89-a771-d9cf1df08c6c","Type":"ContainerDied","Data":"26d50f36774d1424be69e4ff5e46616135f7c06be46f92fac89aba51cd735ec3"}
Sep 11 12:55:48.491484 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:48.491449    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv" event={"ID":"814827b4-7a34-4e89-a771-d9cf1df08c6c","Type":"ContainerStarted","Data":"6d4870dbade32c75075ad3120cae02829d11be951f6b36e2471f5d65bb9b18bd"}
Sep 11 12:55:48.491897 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:48.491576    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:55:48.513744 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:48.513684    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv" podStartSLOduration=3.000205654 podStartE2EDuration="7.513667168s" totalImagesPullingTime="3.463476542s" totalInitContainerRuntime="1.049984972s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:55:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:55:48.512614906 +0000 UTC m=+516.574921864" watchObservedRunningTime="2026-09-11 12:55:48.513667168 +0000 UTC m=+516.575974119"
Sep 11 12:55:59.498789 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:55:59.498757    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cbb4999dc-gw4wv"
Sep 11 12:56:01.853401 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:01.853362    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75775448dd-chzl2"]
Sep 11 12:56:01.868568 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:01.868529    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:01.871722 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:01.871690    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 12:56:01.872222 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:01.872202    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 12:56:01.872402 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:01.872389    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 12:56:01.873903 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:01.873881    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 12:56:01.877260 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:01.877236    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75775448dd-chzl2"]
Sep 11 12:56:01.919487 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:01.919446    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e22da6bd-847e-483d-9e31-7cfe9771817a-tls-certs\") pod \"kube-auth-proxy-75775448dd-chzl2\" (UID: \"e22da6bd-847e-483d-9e31-7cfe9771817a\") " pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:01.919678 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:01.919521    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzlwd\" (UniqueName: \"kubernetes.io/projected/e22da6bd-847e-483d-9e31-7cfe9771817a-kube-api-access-dzlwd\") pod \"kube-auth-proxy-75775448dd-chzl2\" (UID: \"e22da6bd-847e-483d-9e31-7cfe9771817a\") " pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:01.919678 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:01.919583    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e22da6bd-847e-483d-9e31-7cfe9771817a-tmp\") pod \"kube-auth-proxy-75775448dd-chzl2\" (UID: \"e22da6bd-847e-483d-9e31-7cfe9771817a\") " pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:02.020805 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:02.020759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e22da6bd-847e-483d-9e31-7cfe9771817a-tls-certs\") pod \"kube-auth-proxy-75775448dd-chzl2\" (UID: \"e22da6bd-847e-483d-9e31-7cfe9771817a\") " pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:02.021006 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:02.020837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzlwd\" (UniqueName: \"kubernetes.io/projected/e22da6bd-847e-483d-9e31-7cfe9771817a-kube-api-access-dzlwd\") pod \"kube-auth-proxy-75775448dd-chzl2\" (UID: \"e22da6bd-847e-483d-9e31-7cfe9771817a\") " pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:02.021006 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:02.020858    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e22da6bd-847e-483d-9e31-7cfe9771817a-tmp\") pod \"kube-auth-proxy-75775448dd-chzl2\" (UID: \"e22da6bd-847e-483d-9e31-7cfe9771817a\") " pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:02.023219 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:02.023191    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e22da6bd-847e-483d-9e31-7cfe9771817a-tmp\") pod \"kube-auth-proxy-75775448dd-chzl2\" (UID: \"e22da6bd-847e-483d-9e31-7cfe9771817a\") " pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:02.023359 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:02.023339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e22da6bd-847e-483d-9e31-7cfe9771817a-tls-certs\") pod \"kube-auth-proxy-75775448dd-chzl2\" (UID: \"e22da6bd-847e-483d-9e31-7cfe9771817a\") " pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:02.043791 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:02.043757    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzlwd\" (UniqueName: \"kubernetes.io/projected/e22da6bd-847e-483d-9e31-7cfe9771817a-kube-api-access-dzlwd\") pod \"kube-auth-proxy-75775448dd-chzl2\" (UID: \"e22da6bd-847e-483d-9e31-7cfe9771817a\") " pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:02.179941 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:02.179850    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2"
Sep 11 12:56:02.458456 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:02.458426    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75775448dd-chzl2"]
Sep 11 12:56:02.461043 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:56:02.461018    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22da6bd_847e_483d_9e31_7cfe9771817a.slice/crio-716f54569ff89a424805ff5ea20ae5da55a5387e84ca53a5853f86e13fad2173 WatchSource:0}: Error finding container 716f54569ff89a424805ff5ea20ae5da55a5387e84ca53a5853f86e13fad2173: Status 404 returned error can't find the container with id 716f54569ff89a424805ff5ea20ae5da55a5387e84ca53a5853f86e13fad2173
Sep 11 12:56:02.551794 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:02.551750    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2" event={"ID":"e22da6bd-847e-483d-9e31-7cfe9771817a","Type":"ContainerStarted","Data":"716f54569ff89a424805ff5ea20ae5da55a5387e84ca53a5853f86e13fad2173"}
Sep 11 12:56:03.437069 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.437029    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"]
Sep 11 12:56:03.439992 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.439962    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.442749 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.442716    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 12:56:03.442910 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.442716    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 12:56:03.443838 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.443548    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 12:56:03.443838 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.443712    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-n44cd\""
Sep 11 12:56:03.443838 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.443783    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 12:56:03.456105 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.456072    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"]
Sep 11 12:56:03.536488 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.536452    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/adf275ae-da90-4a1a-8c5d-9d08f1393356-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.536716 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.536509    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.536716 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.536542    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.536716 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.536624    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.536716 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.536698    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.536932 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.536767    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prs7v\" (UniqueName: \"kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-kube-api-access-prs7v\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.536932 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.536821    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.637737 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.637699    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.638022 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.637787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/adf275ae-da90-4a1a-8c5d-9d08f1393356-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.638022 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.637932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.638022 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.637983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.638022 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.638021    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.638022 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.638048    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.638350 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.638116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prs7v\" (UniqueName: \"kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-kube-api-access-prs7v\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.638720 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.638666    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.640799 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.640760    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/adf275ae-da90-4a1a-8c5d-9d08f1393356-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.641466 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.641438    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.641862 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.641842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.643281 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.643252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.647589 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.647559    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prs7v\" (UniqueName: \"kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-kube-api-access-prs7v\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.647739 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.647566    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kkqgh\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:03.752938 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:03.752826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:04.046829 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:04.046741    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"]
Sep 11 12:56:05.270240 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:56:05.270201    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadf275ae_da90_4a1a_8c5d_9d08f1393356.slice/crio-64ce2c40e297b1140ef985d1995bdcaa8420c648917898f461da83bb23a15d66 WatchSource:0}: Error finding container 64ce2c40e297b1140ef985d1995bdcaa8420c648917898f461da83bb23a15d66: Status 404 returned error can't find the container with id 64ce2c40e297b1140ef985d1995bdcaa8420c648917898f461da83bb23a15d66
Sep 11 12:56:05.568193 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:05.568090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh" event={"ID":"adf275ae-da90-4a1a-8c5d-9d08f1393356","Type":"ContainerStarted","Data":"64ce2c40e297b1140ef985d1995bdcaa8420c648917898f461da83bb23a15d66"}
Sep 11 12:56:05.572756 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:05.572723    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2" event={"ID":"e22da6bd-847e-483d-9e31-7cfe9771817a","Type":"ContainerStarted","Data":"bbb01a0106aea0299478c716174821d31f24d107af2b868d8f133f5129a9b791"}
Sep 11 12:56:05.590730 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:05.590673    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75775448dd-chzl2" podStartSLOduration=1.729748276 podStartE2EDuration="4.590632289s" totalImagesPullingTime="2.860884013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:05.590149669 +0000 UTC m=+533.652456627" watchObservedRunningTime="2026-09-11 12:56:05.590632289 +0000 UTC m=+533.652939246"
Sep 11 12:56:07.659789 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:07.659749    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 12:56:07.660092 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:07.659817    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 12:56:08.593044 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:08.593006    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh" event={"ID":"adf275ae-da90-4a1a-8c5d-9d08f1393356","Type":"ContainerStarted","Data":"d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1"}
Sep 11 12:56:08.593246 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:08.593214    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:08.595102 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:08.595072    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-kkqgh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 12:56:08.595253 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:08.595124    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh" podUID="adf275ae-da90-4a1a-8c5d-9d08f1393356" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 12:56:08.612724 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:08.612667    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh" podStartSLOduration=3.225536662 podStartE2EDuration="5.612629555s" totalImagesPullingTime="2.387092893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:08.612453165 +0000 UTC m=+536.674760124" watchObservedRunningTime="2026-09-11 12:56:08.612629555 +0000 UTC m=+536.674936512"
Sep 11 12:56:09.598292 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:09.598262    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:12.378282 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.378248    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r64c7"]
Sep 11 12:56:12.382952 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.382930    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r64c7"
Sep 11 12:56:12.385569 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.385545    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 12:56:12.385724 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.385602    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zfhg4\""
Sep 11 12:56:12.390753 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.390411    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r64c7"]
Sep 11 12:56:12.521326 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.521290    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfqdc\" (UniqueName: \"kubernetes.io/projected/c45ed500-705c-46d0-87e4-e7e95a677b30-kube-api-access-qfqdc\") pod \"odh-model-controller-7998c67dc4-r64c7\" (UID: \"c45ed500-705c-46d0-87e4-e7e95a677b30\") " pod="opendatahub/odh-model-controller-7998c67dc4-r64c7"
Sep 11 12:56:12.521513 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.521417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c45ed500-705c-46d0-87e4-e7e95a677b30-cert\") pod \"odh-model-controller-7998c67dc4-r64c7\" (UID: \"c45ed500-705c-46d0-87e4-e7e95a677b30\") " pod="opendatahub/odh-model-controller-7998c67dc4-r64c7"
Sep 11 12:56:12.622488 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.622452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c45ed500-705c-46d0-87e4-e7e95a677b30-cert\") pod \"odh-model-controller-7998c67dc4-r64c7\" (UID: \"c45ed500-705c-46d0-87e4-e7e95a677b30\") " pod="opendatahub/odh-model-controller-7998c67dc4-r64c7"
Sep 11 12:56:12.622488 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.622503    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqdc\" (UniqueName: \"kubernetes.io/projected/c45ed500-705c-46d0-87e4-e7e95a677b30-kube-api-access-qfqdc\") pod \"odh-model-controller-7998c67dc4-r64c7\" (UID: \"c45ed500-705c-46d0-87e4-e7e95a677b30\") " pod="opendatahub/odh-model-controller-7998c67dc4-r64c7"
Sep 11 12:56:12.624970 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.624933    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c45ed500-705c-46d0-87e4-e7e95a677b30-cert\") pod \"odh-model-controller-7998c67dc4-r64c7\" (UID: \"c45ed500-705c-46d0-87e4-e7e95a677b30\") " pod="opendatahub/odh-model-controller-7998c67dc4-r64c7"
Sep 11 12:56:12.634203 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.634125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfqdc\" (UniqueName: \"kubernetes.io/projected/c45ed500-705c-46d0-87e4-e7e95a677b30-kube-api-access-qfqdc\") pod \"odh-model-controller-7998c67dc4-r64c7\" (UID: \"c45ed500-705c-46d0-87e4-e7e95a677b30\") " pod="opendatahub/odh-model-controller-7998c67dc4-r64c7"
Sep 11 12:56:12.694829 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.694790    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r64c7"
Sep 11 12:56:12.977864 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:12.977831    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r64c7"]
Sep 11 12:56:12.980174 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:56:12.980148    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45ed500_705c_46d0_87e4_e7e95a677b30.slice/crio-a0584d3dea753b42495373fafba9b04b315dd25c4c4fe71f32ab368ccaf2bdc6 WatchSource:0}: Error finding container a0584d3dea753b42495373fafba9b04b315dd25c4c4fe71f32ab368ccaf2bdc6: Status 404 returned error can't find the container with id a0584d3dea753b42495373fafba9b04b315dd25c4c4fe71f32ab368ccaf2bdc6
Sep 11 12:56:13.619456 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:13.619411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r64c7" event={"ID":"c45ed500-705c-46d0-87e4-e7e95a677b30","Type":"ContainerStarted","Data":"a0584d3dea753b42495373fafba9b04b315dd25c4c4fe71f32ab368ccaf2bdc6"}
Sep 11 12:56:16.635875 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:16.635823    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r64c7" event={"ID":"c45ed500-705c-46d0-87e4-e7e95a677b30","Type":"ContainerStarted","Data":"a63e3be40cd067a6c6607e4dec924e901b76eb233006ec8d0dd2bd941ce97037"}
Sep 11 12:56:16.636276 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:16.635987    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-r64c7"
Sep 11 12:56:16.654580 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:16.654518    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-r64c7" podStartSLOduration=2.042576261 podStartE2EDuration="4.654500984s" totalImagesPullingTime="2.611924723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:16.65355537 +0000 UTC m=+544.715862329" watchObservedRunningTime="2026-09-11 12:56:16.654500984 +0000 UTC m=+544.716807942"
Sep 11 12:56:19.587961 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.587924    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"]
Sep 11 12:56:19.591353 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.591332    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.606883 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.606851    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"]
Sep 11 12:56:19.684183 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.684139    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.684183 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.684191    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.684411 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.684229    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.684411 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.684282    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.684411 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.684300    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqbbr\" (UniqueName: \"kubernetes.io/projected/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-kube-api-access-sqbbr\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.684411 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.684317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.684411 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.684340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.785925 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.785810    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.786112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.785953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqbbr\" (UniqueName: \"kubernetes.io/projected/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-kube-api-access-sqbbr\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.786112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.785978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.786112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.786003    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.786112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.786066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.786112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.786087    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.786112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.786111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.786939 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.786908    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.788307 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.788278    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.788413 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.788393    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.788467 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.788449    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.788507 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.788459    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.804502 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.804468    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqbbr\" (UniqueName: \"kubernetes.io/projected/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-kube-api-access-sqbbr\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.804620 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.804599    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f63e06ae-e731-41f0-a0a9-8b5e82ac5a26-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-gf682\" (UID: \"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:19.900710 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:19.900610    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:20.181002 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:20.180975    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"]
Sep 11 12:56:20.182555 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:56:20.182527    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf63e06ae_e731_41f0_a0a9_8b5e82ac5a26.slice/crio-7101f59362e08ea7fe9077169ff6266ecb0e2cb681477ed989f3fd16616c1ab5 WatchSource:0}: Error finding container 7101f59362e08ea7fe9077169ff6266ecb0e2cb681477ed989f3fd16616c1ab5: Status 404 returned error can't find the container with id 7101f59362e08ea7fe9077169ff6266ecb0e2cb681477ed989f3fd16616c1ab5
Sep 11 12:56:20.184801 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:20.184767    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 12:56:20.184884 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:20.184831    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 12:56:20.659360 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:20.658499    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682" event={"ID":"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26","Type":"ContainerStarted","Data":"474b207d5c9c0a7a00adbae88800fe91ba2ffafba2257f36da05b3ad120baf3c"}
Sep 11 12:56:20.659360 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:20.658560    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682" event={"ID":"f63e06ae-e731-41f0-a0a9-8b5e82ac5a26","Type":"ContainerStarted","Data":"7101f59362e08ea7fe9077169ff6266ecb0e2cb681477ed989f3fd16616c1ab5"}
Sep 11 12:56:20.659360 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:20.659305    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:20.696556 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:20.696495    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682" podStartSLOduration=1.696471275 podStartE2EDuration="1.696471275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:20.693191531 +0000 UTC m=+548.755498490" watchObservedRunningTime="2026-09-11 12:56:20.696471275 +0000 UTC m=+548.758778233"
Sep 11 12:56:21.666936 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:21.666908    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gf682"
Sep 11 12:56:21.769653 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:21.769608    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"]
Sep 11 12:56:21.769837 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:21.769814    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh" podUID="adf275ae-da90-4a1a-8c5d-9d08f1393356" containerName="discovery" containerID="cri-o://d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1" gracePeriod=30
Sep 11 12:56:22.029369 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.029336    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:22.206029 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.205941    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-kubeconfig\") pod \"adf275ae-da90-4a1a-8c5d-9d08f1393356\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") "
Sep 11 12:56:22.206029 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.206011    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-dns-cert\") pod \"adf275ae-da90-4a1a-8c5d-9d08f1393356\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") "
Sep 11 12:56:22.206255 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.206045    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prs7v\" (UniqueName: \"kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-kube-api-access-prs7v\") pod \"adf275ae-da90-4a1a-8c5d-9d08f1393356\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") "
Sep 11 12:56:22.206255 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.206077    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-cacerts\") pod \"adf275ae-da90-4a1a-8c5d-9d08f1393356\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") "
Sep 11 12:56:22.206255 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.206094    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-token\") pod \"adf275ae-da90-4a1a-8c5d-9d08f1393356\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") "
Sep 11 12:56:22.206255 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.206118    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-ca-configmap\") pod \"adf275ae-da90-4a1a-8c5d-9d08f1393356\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") "
Sep 11 12:56:22.206255 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.206151    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/adf275ae-da90-4a1a-8c5d-9d08f1393356-local-certs\") pod \"adf275ae-da90-4a1a-8c5d-9d08f1393356\" (UID: \"adf275ae-da90-4a1a-8c5d-9d08f1393356\") "
Sep 11 12:56:22.206579 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.206539    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "adf275ae-da90-4a1a-8c5d-9d08f1393356" (UID: "adf275ae-da90-4a1a-8c5d-9d08f1393356"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:56:22.208566 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.208510    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-kube-api-access-prs7v" (OuterVolumeSpecName: "kube-api-access-prs7v") pod "adf275ae-da90-4a1a-8c5d-9d08f1393356" (UID: "adf275ae-da90-4a1a-8c5d-9d08f1393356"). InnerVolumeSpecName "kube-api-access-prs7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:56:22.208566 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.208529    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "adf275ae-da90-4a1a-8c5d-9d08f1393356" (UID: "adf275ae-da90-4a1a-8c5d-9d08f1393356"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:56:22.208731 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.208611    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-cacerts" (OuterVolumeSpecName: "cacerts") pod "adf275ae-da90-4a1a-8c5d-9d08f1393356" (UID: "adf275ae-da90-4a1a-8c5d-9d08f1393356"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:56:22.208731 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.208621    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "adf275ae-da90-4a1a-8c5d-9d08f1393356" (UID: "adf275ae-da90-4a1a-8c5d-9d08f1393356"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:56:22.208805 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.208727    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf275ae-da90-4a1a-8c5d-9d08f1393356-local-certs" (OuterVolumeSpecName: "local-certs") pod "adf275ae-da90-4a1a-8c5d-9d08f1393356" (UID: "adf275ae-da90-4a1a-8c5d-9d08f1393356"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:56:22.208805 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.208737    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-token" (OuterVolumeSpecName: "istio-token") pod "adf275ae-da90-4a1a-8c5d-9d08f1393356" (UID: "adf275ae-da90-4a1a-8c5d-9d08f1393356"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:56:22.307183 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.307151    2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-cacerts\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:56:22.307183 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.307177    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-token\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:56:22.307183 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.307188    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-ca-configmap\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:56:22.307406 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.307197    2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/adf275ae-da90-4a1a-8c5d-9d08f1393356-local-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:56:22.307406 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.307205    2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-kubeconfig\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:56:22.307406 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.307212    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/adf275ae-da90-4a1a-8c5d-9d08f1393356-istio-csr-dns-cert\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:56:22.307406 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.307220    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prs7v\" (UniqueName: \"kubernetes.io/projected/adf275ae-da90-4a1a-8c5d-9d08f1393356-kube-api-access-prs7v\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:56:22.600173 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.600139    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"]
Sep 11 12:56:22.600522 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.600507    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adf275ae-da90-4a1a-8c5d-9d08f1393356" containerName="discovery"
Sep 11 12:56:22.600522 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.600523    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="adf275ae-da90-4a1a-8c5d-9d08f1393356" containerName="discovery"
Sep 11 12:56:22.600622 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.600598    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="adf275ae-da90-4a1a-8c5d-9d08f1393356" containerName="discovery"
Sep 11 12:56:22.603765 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.603744    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:56:22.606357 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.606333    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 11 12:56:22.606495 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.606369    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-v4vmx\""
Sep 11 12:56:22.615310 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.615278    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"]
Sep 11 12:56:22.670440 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.670405    2564 generic.go:356] "Generic (PLEG): container finished" podID="adf275ae-da90-4a1a-8c5d-9d08f1393356" containerID="d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1" exitCode=0
Sep 11 12:56:22.670859 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.670465    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"
Sep 11 12:56:22.670859 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.670462    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh" event={"ID":"adf275ae-da90-4a1a-8c5d-9d08f1393356","Type":"ContainerDied","Data":"d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1"}
Sep 11 12:56:22.670859 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.670579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh" event={"ID":"adf275ae-da90-4a1a-8c5d-9d08f1393356","Type":"ContainerDied","Data":"64ce2c40e297b1140ef985d1995bdcaa8420c648917898f461da83bb23a15d66"}
Sep 11 12:56:22.670859 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.670600    2564 scope.go:117] "RemoveContainer" containerID="d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1"
Sep 11 12:56:22.679954 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.679932    2564 scope.go:117] "RemoveContainer" containerID="d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1"
Sep 11 12:56:22.680307 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:56:22.680283    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1\": container with ID starting with d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1 not found: ID does not exist" containerID="d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1"
Sep 11 12:56:22.680412 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.680313    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1"} err="failed to get container status \"d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1\": rpc error: code = NotFound desc = could not find container \"d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1\": container with ID starting with d310986d8c5e92faf233bebf25531317a2c023e7ed4c3c746a9eb3b2d57295c1 not found: ID does not exist"
Sep 11 12:56:22.688636 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.688605    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"]
Sep 11 12:56:22.693214 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.693188    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kkqgh"]
Sep 11 12:56:22.710292 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.710259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd3ef560-832e-431e-a746-46c3b241ab9a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dksr2\" (UID: \"bd3ef560-832e-431e-a746-46c3b241ab9a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:56:22.710414 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.710305    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78nrq\" (UniqueName: \"kubernetes.io/projected/bd3ef560-832e-431e-a746-46c3b241ab9a-kube-api-access-78nrq\") pod \"llmisvc-controller-manager-777c6bd7d5-dksr2\" (UID: \"bd3ef560-832e-431e-a746-46c3b241ab9a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:56:22.811737 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.811616    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd3ef560-832e-431e-a746-46c3b241ab9a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dksr2\" (UID: \"bd3ef560-832e-431e-a746-46c3b241ab9a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:56:22.811737 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.811706    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78nrq\" (UniqueName: \"kubernetes.io/projected/bd3ef560-832e-431e-a746-46c3b241ab9a-kube-api-access-78nrq\") pod \"llmisvc-controller-manager-777c6bd7d5-dksr2\" (UID: \"bd3ef560-832e-431e-a746-46c3b241ab9a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:56:22.811974 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:56:22.811814    2564 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 11 12:56:22.811974 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:56:22.811886    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3ef560-832e-431e-a746-46c3b241ab9a-cert podName:bd3ef560-832e-431e-a746-46c3b241ab9a nodeName:}" failed. No retries permitted until 2026-09-11 12:56:23.311866353 +0000 UTC m=+551.374173293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bd3ef560-832e-431e-a746-46c3b241ab9a-cert") pod "llmisvc-controller-manager-777c6bd7d5-dksr2" (UID: "bd3ef560-832e-431e-a746-46c3b241ab9a") : secret "llmisvc-webhook-server-cert" not found
Sep 11 12:56:22.825112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:22.825079    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78nrq\" (UniqueName: \"kubernetes.io/projected/bd3ef560-832e-431e-a746-46c3b241ab9a-kube-api-access-78nrq\") pod \"llmisvc-controller-manager-777c6bd7d5-dksr2\" (UID: \"bd3ef560-832e-431e-a746-46c3b241ab9a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:56:23.316557 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:23.316515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd3ef560-832e-431e-a746-46c3b241ab9a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dksr2\" (UID: \"bd3ef560-832e-431e-a746-46c3b241ab9a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:56:23.319079 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:23.319050    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd3ef560-832e-431e-a746-46c3b241ab9a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dksr2\" (UID: \"bd3ef560-832e-431e-a746-46c3b241ab9a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:56:23.513927 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:23.513886    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:56:23.817990 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:23.817965    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"]
Sep 11 12:56:23.821332 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:56:23.821304    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbd3ef560_832e_431e_a746_46c3b241ab9a.slice/crio-53eaa13e03a570ec515b2a79e04a81b81574504340b38222327bfcbbda748ba9 WatchSource:0}: Error finding container 53eaa13e03a570ec515b2a79e04a81b81574504340b38222327bfcbbda748ba9: Status 404 returned error can't find the container with id 53eaa13e03a570ec515b2a79e04a81b81574504340b38222327bfcbbda748ba9
Sep 11 12:56:24.484218 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:24.484188    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adf275ae-da90-4a1a-8c5d-9d08f1393356" path="/var/lib/kubelet/pods/adf275ae-da90-4a1a-8c5d-9d08f1393356/volumes"
Sep 11 12:56:24.684185 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:24.684136    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2" event={"ID":"bd3ef560-832e-431e-a746-46c3b241ab9a","Type":"ContainerStarted","Data":"53eaa13e03a570ec515b2a79e04a81b81574504340b38222327bfcbbda748ba9"}
Sep 11 12:56:26.696355 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:26.696317    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2" event={"ID":"bd3ef560-832e-431e-a746-46c3b241ab9a","Type":"ContainerStarted","Data":"763f55df198e770486d3632f3f7b8505b280c1b0aae7f431c853854ae2d1d3ff"}
Sep 11 12:56:26.696820 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:26.696447    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:56:26.716576 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:26.716462    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2" podStartSLOduration=2.08796008 podStartE2EDuration="4.716441314s" totalImagesPullingTime="2.628481234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:26.716033576 +0000 UTC m=+554.778340536" watchObservedRunningTime="2026-09-11 12:56:26.716441314 +0000 UTC m=+554.778748272"
Sep 11 12:56:27.644968 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:27.644929    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-r64c7"
Sep 11 12:56:57.704024 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:56:57.703994    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dksr2"
Sep 11 12:57:12.426482 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:12.426453    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 12:57:12.426919 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:12.426859    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 12:57:32.010655 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.010603    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd"]
Sep 11 12:57:32.013870 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.013848    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd"
Sep 11 12:57:32.016485 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.016456    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2djhf\""
Sep 11 12:57:32.016614 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.016499    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 11 12:57:32.016707 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.016657    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 12:57:32.016707 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.016699    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 12:57:32.026400 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.026367    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd"]
Sep 11 12:57:32.102352 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.102309    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldkvh\" (UniqueName: \"kubernetes.io/projected/874f2689-6569-45ab-9536-60929b5bcaa6-kube-api-access-ldkvh\") pod \"dns-operator-controller-manager-7b84cd7c9c-qc7zd\" (UID: \"874f2689-6569-45ab-9536-60929b5bcaa6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd"
Sep 11 12:57:32.203614 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.203574    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldkvh\" (UniqueName: \"kubernetes.io/projected/874f2689-6569-45ab-9536-60929b5bcaa6-kube-api-access-ldkvh\") pod \"dns-operator-controller-manager-7b84cd7c9c-qc7zd\" (UID: \"874f2689-6569-45ab-9536-60929b5bcaa6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd"
Sep 11 12:57:32.217707 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.217677    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldkvh\" (UniqueName: \"kubernetes.io/projected/874f2689-6569-45ab-9536-60929b5bcaa6-kube-api-access-ldkvh\") pod \"dns-operator-controller-manager-7b84cd7c9c-qc7zd\" (UID: \"874f2689-6569-45ab-9536-60929b5bcaa6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd"
Sep 11 12:57:32.324272 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.324182    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd"
Sep 11 12:57:32.603932 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:32.603882    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd"]
Sep 11 12:57:32.606608 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:57:32.606579    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod874f2689_6569_45ab_9536_60929b5bcaa6.slice/crio-e46e79ff49235e48d33d469ef51dd601be4fecbc4af0b6552d91f579190d11ca WatchSource:0}: Error finding container e46e79ff49235e48d33d469ef51dd601be4fecbc4af0b6552d91f579190d11ca: Status 404 returned error can't find the container with id e46e79ff49235e48d33d469ef51dd601be4fecbc4af0b6552d91f579190d11ca
Sep 11 12:57:33.010426 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:33.010334    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd" event={"ID":"874f2689-6569-45ab-9536-60929b5bcaa6","Type":"ContainerStarted","Data":"e46e79ff49235e48d33d469ef51dd601be4fecbc4af0b6552d91f579190d11ca"}
Sep 11 12:57:36.028745 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.028701    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd" event={"ID":"874f2689-6569-45ab-9536-60929b5bcaa6","Type":"ContainerStarted","Data":"faba52279a5c442b5d079d2170279fb0145d53038c2b2439cfff08e3b6701a18"}
Sep 11 12:57:36.029118 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.028904    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd"
Sep 11 12:57:36.046315 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.046261    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd" podStartSLOduration=2.443819575 podStartE2EDuration="5.046242583s" totalImagesPullingTime="2.602423008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:36.045372489 +0000 UTC m=+624.107679472" watchObservedRunningTime="2026-09-11 12:57:36.046242583 +0000 UTC m=+624.108549541"
Sep 11 12:57:36.351572 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.351536    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"]
Sep 11 12:57:36.354828 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.354807    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:36.357859 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.357830    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7rmlw\""
Sep 11 12:57:36.382757 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.382718    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"]
Sep 11 12:57:36.443395 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.443357    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbfd07ca-c180-49c0-a64a-33164690e53d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" (UID: \"cbfd07ca-c180-49c0-a64a-33164690e53d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:36.443395 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.443400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ccmz\" (UniqueName: \"kubernetes.io/projected/cbfd07ca-c180-49c0-a64a-33164690e53d-kube-api-access-6ccmz\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" (UID: \"cbfd07ca-c180-49c0-a64a-33164690e53d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:36.544741 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.544700    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbfd07ca-c180-49c0-a64a-33164690e53d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" (UID: \"cbfd07ca-c180-49c0-a64a-33164690e53d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:36.544741 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.544748    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccmz\" (UniqueName: \"kubernetes.io/projected/cbfd07ca-c180-49c0-a64a-33164690e53d-kube-api-access-6ccmz\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" (UID: \"cbfd07ca-c180-49c0-a64a-33164690e53d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:36.545112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.545091    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbfd07ca-c180-49c0-a64a-33164690e53d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" (UID: \"cbfd07ca-c180-49c0-a64a-33164690e53d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:36.558349 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.558311    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ccmz\" (UniqueName: \"kubernetes.io/projected/cbfd07ca-c180-49c0-a64a-33164690e53d-kube-api-access-6ccmz\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" (UID: \"cbfd07ca-c180-49c0-a64a-33164690e53d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:36.665972 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.665877    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:36.945622 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:36.945595    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"]
Sep 11 12:57:36.947538 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:57:36.947514    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbfd07ca_c180_49c0_a64a_33164690e53d.slice/crio-b75a7867d499f68eea8056f17cf75576d19dc32b509c433c35b7ed3847dc0940 WatchSource:0}: Error finding container b75a7867d499f68eea8056f17cf75576d19dc32b509c433c35b7ed3847dc0940: Status 404 returned error can't find the container with id b75a7867d499f68eea8056f17cf75576d19dc32b509c433c35b7ed3847dc0940
Sep 11 12:57:37.035508 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:37.035469    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj" event={"ID":"cbfd07ca-c180-49c0-a64a-33164690e53d","Type":"ContainerStarted","Data":"b75a7867d499f68eea8056f17cf75576d19dc32b509c433c35b7ed3847dc0940"}
Sep 11 12:57:41.215636 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:41.215589    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"]
Sep 11 12:57:41.219729 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:41.219697    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:41.222732 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:41.222701    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-9d9r9\""
Sep 11 12:57:41.237137 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:41.237059    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"]
Sep 11 12:57:41.287887 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:41.287845    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wggbc\" (UniqueName: \"kubernetes.io/projected/4f4709dc-b882-4a6a-b204-956578aeec8a-kube-api-access-wggbc\") pod \"limitador-operator-controller-manager-74495c69f8-9d9fm\" (UID: \"4f4709dc-b882-4a6a-b204-956578aeec8a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:41.388847 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:41.388797    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wggbc\" (UniqueName: \"kubernetes.io/projected/4f4709dc-b882-4a6a-b204-956578aeec8a-kube-api-access-wggbc\") pod \"limitador-operator-controller-manager-74495c69f8-9d9fm\" (UID: \"4f4709dc-b882-4a6a-b204-956578aeec8a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:41.401907 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:41.401868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wggbc\" (UniqueName: \"kubernetes.io/projected/4f4709dc-b882-4a6a-b204-956578aeec8a-kube-api-access-wggbc\") pod \"limitador-operator-controller-manager-74495c69f8-9d9fm\" (UID: \"4f4709dc-b882-4a6a-b204-956578aeec8a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:41.532741 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:41.532656    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:42.635450 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:42.635420    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"]
Sep 11 12:57:42.638061 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:57:42.638025    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4709dc_b882_4a6a_b204_956578aeec8a.slice/crio-721dffba24e41fa33bd3d91b2ba9d71cc701fc17d3718eb8d82923965b72d8d4 WatchSource:0}: Error finding container 721dffba24e41fa33bd3d91b2ba9d71cc701fc17d3718eb8d82923965b72d8d4: Status 404 returned error can't find the container with id 721dffba24e41fa33bd3d91b2ba9d71cc701fc17d3718eb8d82923965b72d8d4
Sep 11 12:57:43.069845 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:43.069740    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj" event={"ID":"cbfd07ca-c180-49c0-a64a-33164690e53d","Type":"ContainerStarted","Data":"fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024"}
Sep 11 12:57:43.070009 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:43.069896    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:43.071247 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:43.071222    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm" event={"ID":"4f4709dc-b882-4a6a-b204-956578aeec8a","Type":"ContainerStarted","Data":"721dffba24e41fa33bd3d91b2ba9d71cc701fc17d3718eb8d82923965b72d8d4"}
Sep 11 12:57:43.092220 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:43.092161    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj" podStartSLOduration=1.598465534 podStartE2EDuration="7.092133986s" totalImagesPullingTime="5.493668452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:43.090620539 +0000 UTC m=+631.152927500" watchObservedRunningTime="2026-09-11 12:57:43.092133986 +0000 UTC m=+631.154440944"
Sep 11 12:57:45.083896 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:45.083857    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm" event={"ID":"4f4709dc-b882-4a6a-b204-956578aeec8a","Type":"ContainerStarted","Data":"6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b"}
Sep 11 12:57:45.084316 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:45.083980    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:45.111579 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:45.111526    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm" podStartSLOduration=2.7187580540000003 podStartE2EDuration="4.11150923s" totalImagesPullingTime="1.392751176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:57:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:45.109868593 +0000 UTC m=+633.172175551" watchObservedRunningTime="2026-09-11 12:57:45.11150923 +0000 UTC m=+633.173816187"
Sep 11 12:57:47.037968 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:47.037932    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qc7zd"
Sep 11 12:57:54.080162 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:54.080126    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:56.013027 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.012989    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"]
Sep 11 12:57:56.016506 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.016488    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:56.039863 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.039827    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"]
Sep 11 12:57:56.091134 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.091102    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:56.118334 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.118301    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr8hq\" (UniqueName: \"kubernetes.io/projected/6f1574df-0a16-415f-9d1a-5ef09862cbd6-kube-api-access-zr8hq\") pod \"kuadrant-operator-controller-manager-5f685c496f-v97pv\" (UID: \"6f1574df-0a16-415f-9d1a-5ef09862cbd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:56.118530 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.118367    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1574df-0a16-415f-9d1a-5ef09862cbd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-v97pv\" (UID: \"6f1574df-0a16-415f-9d1a-5ef09862cbd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:56.210206 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.210169    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"]
Sep 11 12:57:56.210474 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:56.210455    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-zr8hq], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv" podUID="6f1574df-0a16-415f-9d1a-5ef09862cbd6"
Sep 11 12:57:56.219322 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.219289    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1574df-0a16-415f-9d1a-5ef09862cbd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-v97pv\" (UID: \"6f1574df-0a16-415f-9d1a-5ef09862cbd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:56.219450 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.219438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr8hq\" (UniqueName: \"kubernetes.io/projected/6f1574df-0a16-415f-9d1a-5ef09862cbd6-kube-api-access-zr8hq\") pod \"kuadrant-operator-controller-manager-5f685c496f-v97pv\" (UID: \"6f1574df-0a16-415f-9d1a-5ef09862cbd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:56.219733 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.219711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1574df-0a16-415f-9d1a-5ef09862cbd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-v97pv\" (UID: \"6f1574df-0a16-415f-9d1a-5ef09862cbd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:56.243354 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.243311    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr8hq\" (UniqueName: \"kubernetes.io/projected/6f1574df-0a16-415f-9d1a-5ef09862cbd6-kube-api-access-zr8hq\") pod \"kuadrant-operator-controller-manager-5f685c496f-v97pv\" (UID: \"6f1574df-0a16-415f-9d1a-5ef09862cbd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:56.267545 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.267461    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"]
Sep 11 12:57:56.270878 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.270860    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:56.289211 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.289179    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"]
Sep 11 12:57:56.421871 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.421833    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-v2777\" (UID: \"3ccf8092-8b8c-4ead-90e9-4eb3f4845976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:56.422084 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.421888    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v858q\" (UniqueName: \"kubernetes.io/projected/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-kube-api-access-v858q\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-v2777\" (UID: \"3ccf8092-8b8c-4ead-90e9-4eb3f4845976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:56.522617 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.522527    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-v2777\" (UID: \"3ccf8092-8b8c-4ead-90e9-4eb3f4845976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:56.522617 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.522598    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v858q\" (UniqueName: \"kubernetes.io/projected/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-kube-api-access-v858q\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-v2777\" (UID: \"3ccf8092-8b8c-4ead-90e9-4eb3f4845976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:56.522971 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.522950    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-v2777\" (UID: \"3ccf8092-8b8c-4ead-90e9-4eb3f4845976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:56.531400 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.531368    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"]
Sep 11 12:57:56.541747 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.541709    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v858q\" (UniqueName: \"kubernetes.io/projected/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-kube-api-access-v858q\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-v2777\" (UID: \"3ccf8092-8b8c-4ead-90e9-4eb3f4845976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:56.542193 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.542174    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"]
Sep 11 12:57:56.564375 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.564340    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"]
Sep 11 12:57:56.564585 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.564554    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" containerName="manager" containerID="cri-o://fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024" gracePeriod=2
Sep 11 12:57:56.581265 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.581242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:56.591593 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.588832    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"]
Sep 11 12:57:56.601045 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:56.600999    2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"cbfd07ca-c180-49c0-a64a-33164690e53d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\": pods \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:56.611839 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.611750    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"]
Sep 11 12:57:56.615439 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.615406    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"]
Sep 11 12:57:56.616020 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.615999    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" containerName="manager"
Sep 11 12:57:56.616020 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.616021    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" containerName="manager"
Sep 11 12:57:56.616193 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.616113    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" containerName="manager"
Sep 11 12:57:56.619803 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.619778    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:57:56.627090 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.627019    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"]
Sep 11 12:57:56.641667 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.641621    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"]
Sep 11 12:57:56.649758 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.649721    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"]
Sep 11 12:57:56.649923 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.649903    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" containerName="manager" containerID="cri-o://6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b" gracePeriod=2
Sep 11 12:57:56.652327 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.652289    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"]
Sep 11 12:57:56.656157 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.656130    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:57:56.664660 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.661594    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"]
Sep 11 12:57:56.674615 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.674576    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"]
Sep 11 12:57:56.692204 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:56.692150    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:56.724849 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.724811    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/68242c1a-8b41-4abf-9305-b2c995ec2d91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-nm76z\" (UID: \"68242c1a-8b41-4abf-9305-b2c995ec2d91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:57:56.725033 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.724865    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4dcc50f2-5e22-447b-8db2-b010c51afca9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hv7hx\" (UID: \"4dcc50f2-5e22-447b-8db2-b010c51afca9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:57:56.725033 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.724940    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9xwh\" (UniqueName: \"kubernetes.io/projected/68242c1a-8b41-4abf-9305-b2c995ec2d91-kube-api-access-q9xwh\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-nm76z\" (UID: \"68242c1a-8b41-4abf-9305-b2c995ec2d91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:57:56.725033 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.725008    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qfxl\" (UniqueName: \"kubernetes.io/projected/4dcc50f2-5e22-447b-8db2-b010c51afca9-kube-api-access-7qfxl\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hv7hx\" (UID: \"4dcc50f2-5e22-447b-8db2-b010c51afca9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:57:56.744303 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:56.744260    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9d9fm\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:56.749152 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:56.749120    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:56.826411 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.826283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9xwh\" (UniqueName: \"kubernetes.io/projected/68242c1a-8b41-4abf-9305-b2c995ec2d91-kube-api-access-q9xwh\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-nm76z\" (UID: \"68242c1a-8b41-4abf-9305-b2c995ec2d91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:57:56.826411 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.826351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qfxl\" (UniqueName: \"kubernetes.io/projected/4dcc50f2-5e22-447b-8db2-b010c51afca9-kube-api-access-7qfxl\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hv7hx\" (UID: \"4dcc50f2-5e22-447b-8db2-b010c51afca9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:57:56.826560 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.826462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/68242c1a-8b41-4abf-9305-b2c995ec2d91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-nm76z\" (UID: \"68242c1a-8b41-4abf-9305-b2c995ec2d91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:57:56.826560 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.826498    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4dcc50f2-5e22-447b-8db2-b010c51afca9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hv7hx\" (UID: \"4dcc50f2-5e22-447b-8db2-b010c51afca9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:57:56.826983 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.826947    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4dcc50f2-5e22-447b-8db2-b010c51afca9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hv7hx\" (UID: \"4dcc50f2-5e22-447b-8db2-b010c51afca9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:57:56.827093 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.827067    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/68242c1a-8b41-4abf-9305-b2c995ec2d91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-nm76z\" (UID: \"68242c1a-8b41-4abf-9305-b2c995ec2d91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:57:56.848075 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.848046    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qfxl\" (UniqueName: \"kubernetes.io/projected/4dcc50f2-5e22-447b-8db2-b010c51afca9-kube-api-access-7qfxl\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hv7hx\" (UID: \"4dcc50f2-5e22-447b-8db2-b010c51afca9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:57:56.848422 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.848400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9xwh\" (UniqueName: \"kubernetes.io/projected/68242c1a-8b41-4abf-9305-b2c995ec2d91-kube-api-access-q9xwh\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-nm76z\" (UID: \"68242c1a-8b41-4abf-9305-b2c995ec2d91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:57:56.888169 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.888140    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:56.892189 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.892159    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:56.896764 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:56.896719    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9d9fm\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:56.900388 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:56.900351    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:56.902777 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:56.902751    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9d9fm\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:56.905254 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:56.905228    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:56.949598 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:56.949566    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:57:57.028099 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.028068    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ccmz\" (UniqueName: \"kubernetes.io/projected/cbfd07ca-c180-49c0-a64a-33164690e53d-kube-api-access-6ccmz\") pod \"cbfd07ca-c180-49c0-a64a-33164690e53d\" (UID: \"cbfd07ca-c180-49c0-a64a-33164690e53d\") "
Sep 11 12:57:57.028531 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.028112    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbfd07ca-c180-49c0-a64a-33164690e53d-extensions-socket-volume\") pod \"cbfd07ca-c180-49c0-a64a-33164690e53d\" (UID: \"cbfd07ca-c180-49c0-a64a-33164690e53d\") "
Sep 11 12:57:57.028531 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.028167    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wggbc\" (UniqueName: \"kubernetes.io/projected/4f4709dc-b882-4a6a-b204-956578aeec8a-kube-api-access-wggbc\") pod \"4f4709dc-b882-4a6a-b204-956578aeec8a\" (UID: \"4f4709dc-b882-4a6a-b204-956578aeec8a\") "
Sep 11 12:57:57.028793 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.028762    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbfd07ca-c180-49c0-a64a-33164690e53d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cbfd07ca-c180-49c0-a64a-33164690e53d" (UID: "cbfd07ca-c180-49c0-a64a-33164690e53d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:57:57.030470 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.030444    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f4709dc-b882-4a6a-b204-956578aeec8a-kube-api-access-wggbc" (OuterVolumeSpecName: "kube-api-access-wggbc") pod "4f4709dc-b882-4a6a-b204-956578aeec8a" (UID: "4f4709dc-b882-4a6a-b204-956578aeec8a"). InnerVolumeSpecName "kube-api-access-wggbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:57:57.030470 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.030451    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbfd07ca-c180-49c0-a64a-33164690e53d-kube-api-access-6ccmz" (OuterVolumeSpecName: "kube-api-access-6ccmz") pod "cbfd07ca-c180-49c0-a64a-33164690e53d" (UID: "cbfd07ca-c180-49c0-a64a-33164690e53d"). InnerVolumeSpecName "kube-api-access-6ccmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:57:57.035655 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.035609    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:57:57.129244 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.129206    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wggbc\" (UniqueName: \"kubernetes.io/projected/4f4709dc-b882-4a6a-b204-956578aeec8a-kube-api-access-wggbc\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:57:57.129244 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.129241    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ccmz\" (UniqueName: \"kubernetes.io/projected/cbfd07ca-c180-49c0-a64a-33164690e53d-kube-api-access-6ccmz\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:57:57.129244 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.129252    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbfd07ca-c180-49c0-a64a-33164690e53d-extensions-socket-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:57:57.143558 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.143520    2564 generic.go:356] "Generic (PLEG): container finished" podID="cbfd07ca-c180-49c0-a64a-33164690e53d" containerID="fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024" exitCode=0
Sep 11 12:57:57.143752 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.143567    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:57.143752 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.143611    2564 scope.go:117] "RemoveContainer" containerID="fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024"
Sep 11 12:57:57.144998 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.144976    2564 generic.go:356] "Generic (PLEG): container finished" podID="4f4709dc-b882-4a6a-b204-956578aeec8a" containerID="6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b" exitCode=0
Sep 11 12:57:57.145093 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.145023    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:57.145240 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.145222    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:57.146382 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.146356    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9d9fm\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:57.149470 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.149430    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:57.153350 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.153100    2564 scope.go:117] "RemoveContainer" containerID="fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024"
Sep 11 12:57:57.153350 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.153107    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9d9fm\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:57.153498 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.153468    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024\": container with ID starting with fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024 not found: ID does not exist" containerID="fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024"
Sep 11 12:57:57.153538 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.153510    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024"} err="failed to get container status \"fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024\": rpc error: code = NotFound desc = could not find container \"fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024\": container with ID starting with fe4ea5937aaa675b3212981afed945c3f72b77582b6ffc1ece239df463720024 not found: ID does not exist"
Sep 11 12:57:57.153538 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.153529    2564 scope.go:117] "RemoveContainer" containerID="6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b"
Sep 11 12:57:57.154709 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.154598    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:57.155899 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.155870    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-v97pv\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="6f1574df-0a16-415f-9d1a-5ef09862cbd6" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:57.157897 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.157870    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:57.160145 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.160118    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:57.162237 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.162214    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9d9fm\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:57.162659 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.162629    2564 scope.go:117] "RemoveContainer" containerID="6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b"
Sep 11 12:57:57.162955 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.162934    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b\": container with ID starting with 6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b not found: ID does not exist" containerID="6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b"
Sep 11 12:57:57.163046 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.162960    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b"} err="failed to get container status \"6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b\": rpc error: code = NotFound desc = could not find container \"6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b\": container with ID starting with 6ca2a9f048c584fbac1eb40a3f0c058aa596c62cd39c60839a51bff3af51194b not found: ID does not exist"
Sep 11 12:57:57.168376 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.168345    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-v97pv\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="6f1574df-0a16-415f-9d1a-5ef09862cbd6" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:57.230235 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.230203    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1574df-0a16-415f-9d1a-5ef09862cbd6-extensions-socket-volume\") pod \"6f1574df-0a16-415f-9d1a-5ef09862cbd6\" (UID: \"6f1574df-0a16-415f-9d1a-5ef09862cbd6\") "
Sep 11 12:57:57.230727 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.230274    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr8hq\" (UniqueName: \"kubernetes.io/projected/6f1574df-0a16-415f-9d1a-5ef09862cbd6-kube-api-access-zr8hq\") pod \"6f1574df-0a16-415f-9d1a-5ef09862cbd6\" (UID: \"6f1574df-0a16-415f-9d1a-5ef09862cbd6\") "
Sep 11 12:57:57.230727 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.230525    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1574df-0a16-415f-9d1a-5ef09862cbd6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6f1574df-0a16-415f-9d1a-5ef09862cbd6" (UID: "6f1574df-0a16-415f-9d1a-5ef09862cbd6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:57:57.230727 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.230655    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1574df-0a16-415f-9d1a-5ef09862cbd6-extensions-socket-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:57:57.232568 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.232545    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f1574df-0a16-415f-9d1a-5ef09862cbd6-kube-api-access-zr8hq" (OuterVolumeSpecName: "kube-api-access-zr8hq") pod "6f1574df-0a16-415f-9d1a-5ef09862cbd6" (UID: "6f1574df-0a16-415f-9d1a-5ef09862cbd6"). InnerVolumeSpecName "kube-api-access-zr8hq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:57:57.232689 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.232616    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"]
Sep 11 12:57:57.239535 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:57:57.239507    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dcc50f2_5e22_447b_8db2_b010c51afca9.slice/crio-b6578ab06aa94ba38389a2d526e217fa8ca9f50a6bdbcb76142e59c01d1bf4bc WatchSource:0}: Error finding container b6578ab06aa94ba38389a2d526e217fa8ca9f50a6bdbcb76142e59c01d1bf4bc: Status 404 returned error can't find the container with id b6578ab06aa94ba38389a2d526e217fa8ca9f50a6bdbcb76142e59c01d1bf4bc
Sep 11 12:57:57.313971 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.313944    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"]
Sep 11 12:57:57.315801 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:57:57.315772    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68242c1a_8b41_4abf_9305_b2c995ec2d91.slice/crio-5157b72071a343463f55de28ee579993e99a6ca32d7acfaf2f550dfa666952ac WatchSource:0}: Error finding container 5157b72071a343463f55de28ee579993e99a6ca32d7acfaf2f550dfa666952ac: Status 404 returned error can't find the container with id 5157b72071a343463f55de28ee579993e99a6ca32d7acfaf2f550dfa666952ac
Sep 11 12:57:57.331897 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:57.331874    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr8hq\" (UniqueName: \"kubernetes.io/projected/6f1574df-0a16-415f-9d1a-5ef09862cbd6-kube-api-access-zr8hq\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:57:57.592177 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.592136    2564 log.go:32] "RunPodSandbox from runtime service failed" err=<
Sep 11 12:57:57.592177 ip-10-0-136-55 kubenswrapper[2564]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-v2777_kuadrant-system_3ccf8092-8b8c-4ead-90e9-4eb3f4845976_0(36d705e2e94377c4d1ec6405ddb7d776ca8e9ca67313c4d21a2e986714472df8): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-v2777 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"36d705e2e94377c4d1ec6405ddb7d776ca8e9ca67313c4d21a2e986714472df8" Netns:"/var/run/netns/bebd514f-5795-4941-ac5c-77c8890b0710" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-v2777;K8S_POD_INFRA_CONTAINER_ID=36d705e2e94377c4d1ec6405ddb7d776ca8e9ca67313c4d21a2e986714472df8;K8S_POD_UID=3ccf8092-8b8c-4ead-90e9-4eb3f4845976" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777/3ccf8092-8b8c-4ead-90e9-4eb3f4845976:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777 36d705e2e94377c4d1ec6405ddb7d776ca8e9ca67313c4d21a2e986714472df8 network default NAD default NAD key default] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-7978cbdfbc-v2777" not found
Sep 11 12:57:57.592177 ip-10-0-136-55 kubenswrapper[2564]:         '
Sep 11 12:57:57.592177 ip-10-0-136-55 kubenswrapper[2564]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 11 12:57:57.592177 ip-10-0-136-55 kubenswrapper[2564]:  >
Sep 11 12:57:57.592422 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:57.592202    2564 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Sep 11 12:57:57.592422 ip-10-0-136-55 kubenswrapper[2564]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-v2777_kuadrant-system_3ccf8092-8b8c-4ead-90e9-4eb3f4845976_0(36d705e2e94377c4d1ec6405ddb7d776ca8e9ca67313c4d21a2e986714472df8): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-v2777 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"36d705e2e94377c4d1ec6405ddb7d776ca8e9ca67313c4d21a2e986714472df8" Netns:"/var/run/netns/bebd514f-5795-4941-ac5c-77c8890b0710" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-v2777;K8S_POD_INFRA_CONTAINER_ID=36d705e2e94377c4d1ec6405ddb7d776ca8e9ca67313c4d21a2e986714472df8;K8S_POD_UID=3ccf8092-8b8c-4ead-90e9-4eb3f4845976" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777/3ccf8092-8b8c-4ead-90e9-4eb3f4845976:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777 36d705e2e94377c4d1ec6405ddb7d776ca8e9ca67313c4d21a2e986714472df8 network default NAD default NAD key default] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-7978cbdfbc-v2777" not found
Sep 11 12:57:57.592422 ip-10-0-136-55 kubenswrapper[2564]:         '
Sep 11 12:57:57.592422 ip-10-0-136-55 kubenswrapper[2564]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 11 12:57:57.592422 ip-10-0-136-55 kubenswrapper[2564]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:58.154925 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.154889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx" event={"ID":"4dcc50f2-5e22-447b-8db2-b010c51afca9","Type":"ContainerStarted","Data":"7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244"}
Sep 11 12:57:58.155387 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.154951    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:57:58.155387 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.154977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx" event={"ID":"4dcc50f2-5e22-447b-8db2-b010c51afca9","Type":"ContainerStarted","Data":"b6578ab06aa94ba38389a2d526e217fa8ca9f50a6bdbcb76142e59c01d1bf4bc"}
Sep 11 12:57:58.156912 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.156889    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:58.156912 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.156899    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:58.156912 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.156883    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z" event={"ID":"68242c1a-8b41-4abf-9305-b2c995ec2d91","Type":"ContainerStarted","Data":"0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b"}
Sep 11 12:57:58.157153 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.156946    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z" event={"ID":"68242c1a-8b41-4abf-9305-b2c995ec2d91","Type":"ContainerStarted","Data":"5157b72071a343463f55de28ee579993e99a6ca32d7acfaf2f550dfa666952ac"}
Sep 11 12:57:58.157153 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.156995    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:57:58.159156 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:58.159124    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9d9fm\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:58.161202 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:58.161158    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-v97pv\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="6f1574df-0a16-415f-9d1a-5ef09862cbd6" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:58.163182 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:58.163156    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:58.165555 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.165532    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:58.186409 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.186356    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx" podStartSLOduration=2.1863366539999998 podStartE2EDuration="2.186336654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:58.185134888 +0000 UTC m=+646.247441870" watchObservedRunningTime="2026-09-11 12:57:58.186336654 +0000 UTC m=+646.248643613"
Sep 11 12:57:58.187121 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:58.187090    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-2kkfj\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2kkfj"
Sep 11 12:57:58.206476 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.206387    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z" podStartSLOduration=2.206366738 podStartE2EDuration="2.206366738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:58.205175993 +0000 UTC m=+646.267482973" watchObservedRunningTime="2026-09-11 12:57:58.206366738 +0000 UTC m=+646.268673697"
Sep 11 12:57:58.207703 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:58.207667    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-v2777\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="3ccf8092-8b8c-4ead-90e9-4eb3f4845976" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:58.209948 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:58.209918    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9d9fm\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9d9fm"
Sep 11 12:57:58.212015 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:58.211987    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-v97pv\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="6f1574df-0a16-415f-9d1a-5ef09862cbd6" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v97pv"
Sep 11 12:57:58.241186 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.240399    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v858q\" (UniqueName: \"kubernetes.io/projected/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-kube-api-access-v858q\") pod \"3ccf8092-8b8c-4ead-90e9-4eb3f4845976\" (UID: \"3ccf8092-8b8c-4ead-90e9-4eb3f4845976\") "
Sep 11 12:57:58.241186 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.240515    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-extensions-socket-volume\") pod \"3ccf8092-8b8c-4ead-90e9-4eb3f4845976\" (UID: \"3ccf8092-8b8c-4ead-90e9-4eb3f4845976\") "
Sep 11 12:57:58.241186 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.240932    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3ccf8092-8b8c-4ead-90e9-4eb3f4845976" (UID: "3ccf8092-8b8c-4ead-90e9-4eb3f4845976"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:57:58.243591 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.243553    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-kube-api-access-v858q" (OuterVolumeSpecName: "kube-api-access-v858q") pod "3ccf8092-8b8c-4ead-90e9-4eb3f4845976" (UID: "3ccf8092-8b8c-4ead-90e9-4eb3f4845976"). InnerVolumeSpecName "kube-api-access-v858q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:57:58.342347 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.342305    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v858q\" (UniqueName: \"kubernetes.io/projected/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-kube-api-access-v858q\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:57:58.342347 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.342346    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccf8092-8b8c-4ead-90e9-4eb3f4845976-extensions-socket-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:57:58.483895 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.483819    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ccf8092-8b8c-4ead-90e9-4eb3f4845976" path="/var/lib/kubelet/pods/3ccf8092-8b8c-4ead-90e9-4eb3f4845976/volumes"
Sep 11 12:57:58.484099 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.484082    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" path="/var/lib/kubelet/pods/4f4709dc-b882-4a6a-b204-956578aeec8a/volumes"
Sep 11 12:57:58.484481 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.484467    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f1574df-0a16-415f-9d1a-5ef09862cbd6" path="/var/lib/kubelet/pods/6f1574df-0a16-415f-9d1a-5ef09862cbd6/volumes"
Sep 11 12:57:58.484763 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:58.484750    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbfd07ca-c180-49c0-a64a-33164690e53d" path="/var/lib/kubelet/pods/cbfd07ca-c180-49c0-a64a-33164690e53d/volumes"
Sep 11 12:57:59.161898 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:57:59.161867    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:57:59.166237 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:57:59.166209    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-v2777\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="3ccf8092-8b8c-4ead-90e9-4eb3f4845976" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-v2777"
Sep 11 12:58:09.163981 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.163949    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:58:09.164687 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.164018    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:58:09.260473 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.260437    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"]
Sep 11 12:58:09.260724 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.260696    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx" podUID="4dcc50f2-5e22-447b-8db2-b010c51afca9" containerName="manager" containerID="cri-o://7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244" gracePeriod=10
Sep 11 12:58:09.502533 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.502504    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"]
Sep 11 12:58:09.502913 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.502901    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" containerName="manager"
Sep 11 12:58:09.502967 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.502915    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" containerName="manager"
Sep 11 12:58:09.503003 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.502975    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4709dc-b882-4a6a-b204-956578aeec8a" containerName="manager"
Sep 11 12:58:09.505007 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.504986    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 12:58:09.509796 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.509773    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:58:09.524156 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.524130    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"]
Sep 11 12:58:09.642268 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.642235    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qfxl\" (UniqueName: \"kubernetes.io/projected/4dcc50f2-5e22-447b-8db2-b010c51afca9-kube-api-access-7qfxl\") pod \"4dcc50f2-5e22-447b-8db2-b010c51afca9\" (UID: \"4dcc50f2-5e22-447b-8db2-b010c51afca9\") "
Sep 11 12:58:09.642442 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.642367    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4dcc50f2-5e22-447b-8db2-b010c51afca9-extensions-socket-volume\") pod \"4dcc50f2-5e22-447b-8db2-b010c51afca9\" (UID: \"4dcc50f2-5e22-447b-8db2-b010c51afca9\") "
Sep 11 12:58:09.642505 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.642462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z4vm\" (UniqueName: \"kubernetes.io/projected/341e7875-a1bd-4402-93db-6d4131cbaed3-kube-api-access-4z4vm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cwpj8\" (UID: \"341e7875-a1bd-4402-93db-6d4131cbaed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 12:58:09.642621 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.642600    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/341e7875-a1bd-4402-93db-6d4131cbaed3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cwpj8\" (UID: \"341e7875-a1bd-4402-93db-6d4131cbaed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 12:58:09.642780 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.642758    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dcc50f2-5e22-447b-8db2-b010c51afca9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4dcc50f2-5e22-447b-8db2-b010c51afca9" (UID: "4dcc50f2-5e22-447b-8db2-b010c51afca9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:58:09.644310 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.644289    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dcc50f2-5e22-447b-8db2-b010c51afca9-kube-api-access-7qfxl" (OuterVolumeSpecName: "kube-api-access-7qfxl") pod "4dcc50f2-5e22-447b-8db2-b010c51afca9" (UID: "4dcc50f2-5e22-447b-8db2-b010c51afca9"). InnerVolumeSpecName "kube-api-access-7qfxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:58:09.743622 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.743529    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/341e7875-a1bd-4402-93db-6d4131cbaed3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cwpj8\" (UID: \"341e7875-a1bd-4402-93db-6d4131cbaed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 12:58:09.743622 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.743598    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4vm\" (UniqueName: \"kubernetes.io/projected/341e7875-a1bd-4402-93db-6d4131cbaed3-kube-api-access-4z4vm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cwpj8\" (UID: \"341e7875-a1bd-4402-93db-6d4131cbaed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 12:58:09.743837 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.743665    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4dcc50f2-5e22-447b-8db2-b010c51afca9-extensions-socket-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:58:09.743837 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.743679    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qfxl\" (UniqueName: \"kubernetes.io/projected/4dcc50f2-5e22-447b-8db2-b010c51afca9-kube-api-access-7qfxl\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:58:09.743969 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.743948    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/341e7875-a1bd-4402-93db-6d4131cbaed3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cwpj8\" (UID: \"341e7875-a1bd-4402-93db-6d4131cbaed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 12:58:09.753055 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.753023    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z4vm\" (UniqueName: \"kubernetes.io/projected/341e7875-a1bd-4402-93db-6d4131cbaed3-kube-api-access-4z4vm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cwpj8\" (UID: \"341e7875-a1bd-4402-93db-6d4131cbaed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 12:58:09.817059 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:09.817018    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 12:58:10.097833 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.097801    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"]
Sep 11 12:58:10.099764 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:58:10.099739    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341e7875_a1bd_4402_93db_6d4131cbaed3.slice/crio-343f9e4d9cc952eaa895177be0ccfbdd15ddb36e350a83d08d4d9173f1ecfafe WatchSource:0}: Error finding container 343f9e4d9cc952eaa895177be0ccfbdd15ddb36e350a83d08d4d9173f1ecfafe: Status 404 returned error can't find the container with id 343f9e4d9cc952eaa895177be0ccfbdd15ddb36e350a83d08d4d9173f1ecfafe
Sep 11 12:58:10.219674 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.219623    2564 generic.go:356] "Generic (PLEG): container finished" podID="4dcc50f2-5e22-447b-8db2-b010c51afca9" containerID="7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244" exitCode=0
Sep 11 12:58:10.219674 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.219680    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"
Sep 11 12:58:10.220178 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.219711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx" event={"ID":"4dcc50f2-5e22-447b-8db2-b010c51afca9","Type":"ContainerDied","Data":"7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244"}
Sep 11 12:58:10.220178 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.219759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx" event={"ID":"4dcc50f2-5e22-447b-8db2-b010c51afca9","Type":"ContainerDied","Data":"b6578ab06aa94ba38389a2d526e217fa8ca9f50a6bdbcb76142e59c01d1bf4bc"}
Sep 11 12:58:10.220178 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.219782    2564 scope.go:117] "RemoveContainer" containerID="7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244"
Sep 11 12:58:10.221841 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.221814    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8" event={"ID":"341e7875-a1bd-4402-93db-6d4131cbaed3","Type":"ContainerStarted","Data":"fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985"}
Sep 11 12:58:10.221968 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.221864    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8" event={"ID":"341e7875-a1bd-4402-93db-6d4131cbaed3","Type":"ContainerStarted","Data":"343f9e4d9cc952eaa895177be0ccfbdd15ddb36e350a83d08d4d9173f1ecfafe"}
Sep 11 12:58:10.222036 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.221966    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 12:58:10.228880 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.228862    2564 scope.go:117] "RemoveContainer" containerID="7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244"
Sep 11 12:58:10.229176 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:58:10.229156    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244\": container with ID starting with 7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244 not found: ID does not exist" containerID="7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244"
Sep 11 12:58:10.229235 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.229184    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244"} err="failed to get container status \"7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244\": rpc error: code = NotFound desc = could not find container \"7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244\": container with ID starting with 7a356faa5e5040977d295e2a962cb7c326fc6e007074869ec1983e31064ae244 not found: ID does not exist"
Sep 11 12:58:10.244476 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.244424    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8" podStartSLOduration=1.244403523 podStartE2EDuration="1.244403523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:58:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:58:10.24279094 +0000 UTC m=+658.305097899" watchObservedRunningTime="2026-09-11 12:58:10.244403523 +0000 UTC m=+658.306710481"
Sep 11 12:58:10.260012 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.259979    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"]
Sep 11 12:58:10.263982 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.263954    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hv7hx"]
Sep 11 12:58:10.483065 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:10.482979    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dcc50f2-5e22-447b-8db2-b010c51afca9" path="/var/lib/kubelet/pods/4dcc50f2-5e22-447b-8db2-b010c51afca9/volumes"
Sep 11 12:58:21.230692 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:21.230633    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 12:58:21.289187 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:21.289145    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"]
Sep 11 12:58:21.289461 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:21.289431    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z" podUID="68242c1a-8b41-4abf-9305-b2c995ec2d91" containerName="manager" containerID="cri-o://0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b" gracePeriod=10
Sep 11 12:58:21.547454 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:21.547425    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:58:21.644183 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:21.644149    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/68242c1a-8b41-4abf-9305-b2c995ec2d91-extensions-socket-volume\") pod \"68242c1a-8b41-4abf-9305-b2c995ec2d91\" (UID: \"68242c1a-8b41-4abf-9305-b2c995ec2d91\") "
Sep 11 12:58:21.644345 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:21.644194    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9xwh\" (UniqueName: \"kubernetes.io/projected/68242c1a-8b41-4abf-9305-b2c995ec2d91-kube-api-access-q9xwh\") pod \"68242c1a-8b41-4abf-9305-b2c995ec2d91\" (UID: \"68242c1a-8b41-4abf-9305-b2c995ec2d91\") "
Sep 11 12:58:21.644566 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:21.644546    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68242c1a-8b41-4abf-9305-b2c995ec2d91-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "68242c1a-8b41-4abf-9305-b2c995ec2d91" (UID: "68242c1a-8b41-4abf-9305-b2c995ec2d91"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:58:21.646290 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:21.646268    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68242c1a-8b41-4abf-9305-b2c995ec2d91-kube-api-access-q9xwh" (OuterVolumeSpecName: "kube-api-access-q9xwh") pod "68242c1a-8b41-4abf-9305-b2c995ec2d91" (UID: "68242c1a-8b41-4abf-9305-b2c995ec2d91"). InnerVolumeSpecName "kube-api-access-q9xwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:58:21.745161 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:21.745125    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/68242c1a-8b41-4abf-9305-b2c995ec2d91-extensions-socket-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:58:21.745161 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:21.745155    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9xwh\" (UniqueName: \"kubernetes.io/projected/68242c1a-8b41-4abf-9305-b2c995ec2d91-kube-api-access-q9xwh\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:58:22.288160 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:22.288114    2564 generic.go:356] "Generic (PLEG): container finished" podID="68242c1a-8b41-4abf-9305-b2c995ec2d91" containerID="0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b" exitCode=0
Sep 11 12:58:22.288673 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:22.288279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z" event={"ID":"68242c1a-8b41-4abf-9305-b2c995ec2d91","Type":"ContainerDied","Data":"0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b"}
Sep 11 12:58:22.288673 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:22.288361    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z" event={"ID":"68242c1a-8b41-4abf-9305-b2c995ec2d91","Type":"ContainerDied","Data":"5157b72071a343463f55de28ee579993e99a6ca32d7acfaf2f550dfa666952ac"}
Sep 11 12:58:22.288673 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:22.288403    2564 scope.go:117] "RemoveContainer" containerID="0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b"
Sep 11 12:58:22.288673 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:22.288597    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"
Sep 11 12:58:22.299680 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:22.299635    2564 scope.go:117] "RemoveContainer" containerID="0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b"
Sep 11 12:58:22.300011 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:58:22.299988    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b\": container with ID starting with 0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b not found: ID does not exist" containerID="0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b"
Sep 11 12:58:22.300075 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:22.300021    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b"} err="failed to get container status \"0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b\": rpc error: code = NotFound desc = could not find container \"0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b\": container with ID starting with 0abf1517fdba71ba7d900049438f4566f492dae2eafa8dbba84a70114be6e90b not found: ID does not exist"
Sep 11 12:58:22.314490 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:22.314455    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"]
Sep 11 12:58:22.326250 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:22.326213    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-nm76z"]
Sep 11 12:58:22.483269 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:22.483232    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68242c1a-8b41-4abf-9305-b2c995ec2d91" path="/var/lib/kubelet/pods/68242c1a-8b41-4abf-9305-b2c995ec2d91/volumes"
Sep 11 12:58:37.182436 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.182385    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lkcfx"]
Sep 11 12:58:37.182999 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.182792    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68242c1a-8b41-4abf-9305-b2c995ec2d91" containerName="manager"
Sep 11 12:58:37.182999 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.182806    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="68242c1a-8b41-4abf-9305-b2c995ec2d91" containerName="manager"
Sep 11 12:58:37.182999 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.182827    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dcc50f2-5e22-447b-8db2-b010c51afca9" containerName="manager"
Sep 11 12:58:37.182999 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.182836    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dcc50f2-5e22-447b-8db2-b010c51afca9" containerName="manager"
Sep 11 12:58:37.182999 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.182916    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dcc50f2-5e22-447b-8db2-b010c51afca9" containerName="manager"
Sep 11 12:58:37.182999 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.182931    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="68242c1a-8b41-4abf-9305-b2c995ec2d91" containerName="manager"
Sep 11 12:58:37.186144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.186121    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:37.188913 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.188890    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 12:58:37.189054 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.189038    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bbh28\""
Sep 11 12:58:37.194760 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.194730    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lkcfx"]
Sep 11 12:58:37.280715 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.280678    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lkcfx"]
Sep 11 12:58:37.283098 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.283070    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcqlw\" (UniqueName: \"kubernetes.io/projected/0761664b-d44f-4b7d-856a-ec4c94695340-kube-api-access-zcqlw\") pod \"limitador-limitador-8588749977-lkcfx\" (UID: \"0761664b-d44f-4b7d-856a-ec4c94695340\") " pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:37.283251 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.283109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0761664b-d44f-4b7d-856a-ec4c94695340-config-file\") pod \"limitador-limitador-8588749977-lkcfx\" (UID: \"0761664b-d44f-4b7d-856a-ec4c94695340\") " pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:37.384608 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.384572    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcqlw\" (UniqueName: \"kubernetes.io/projected/0761664b-d44f-4b7d-856a-ec4c94695340-kube-api-access-zcqlw\") pod \"limitador-limitador-8588749977-lkcfx\" (UID: \"0761664b-d44f-4b7d-856a-ec4c94695340\") " pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:37.384828 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.384618    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0761664b-d44f-4b7d-856a-ec4c94695340-config-file\") pod \"limitador-limitador-8588749977-lkcfx\" (UID: \"0761664b-d44f-4b7d-856a-ec4c94695340\") " pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:37.385262 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.385242    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0761664b-d44f-4b7d-856a-ec4c94695340-config-file\") pod \"limitador-limitador-8588749977-lkcfx\" (UID: \"0761664b-d44f-4b7d-856a-ec4c94695340\") " pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:37.398949 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.398910    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcqlw\" (UniqueName: \"kubernetes.io/projected/0761664b-d44f-4b7d-856a-ec4c94695340-kube-api-access-zcqlw\") pod \"limitador-limitador-8588749977-lkcfx\" (UID: \"0761664b-d44f-4b7d-856a-ec4c94695340\") " pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:37.496929 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.496827    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:37.768848 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.768819    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lkcfx"]
Sep 11 12:58:37.771022 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:58:37.770998    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0761664b_d44f_4b7d_856a_ec4c94695340.slice/crio-49f26287d732c0d1fd1efeaa78eb1667e8c3b7629e58496648fb33890bf25590 WatchSource:0}: Error finding container 49f26287d732c0d1fd1efeaa78eb1667e8c3b7629e58496648fb33890bf25590: Status 404 returned error can't find the container with id 49f26287d732c0d1fd1efeaa78eb1667e8c3b7629e58496648fb33890bf25590
Sep 11 12:58:37.772972 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:37.772954    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:58:38.145183 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:38.145147    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-t92hf"]
Sep 11 12:58:38.149800 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:38.149778    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-t92hf"
Sep 11 12:58:38.152261 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:38.152239    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gwf2j\""
Sep 11 12:58:38.156029 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:38.156005    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-t92hf"]
Sep 11 12:58:38.293224 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:38.293187    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqgxp\" (UniqueName: \"kubernetes.io/projected/9fa02b3d-6bfc-4c28-bb1f-1f41a2693781-kube-api-access-dqgxp\") pod \"authorino-59c974bd48-t92hf\" (UID: \"9fa02b3d-6bfc-4c28-bb1f-1f41a2693781\") " pod="kuadrant-system/authorino-59c974bd48-t92hf"
Sep 11 12:58:38.373216 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:38.373184    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx" event={"ID":"0761664b-d44f-4b7d-856a-ec4c94695340","Type":"ContainerStarted","Data":"49f26287d732c0d1fd1efeaa78eb1667e8c3b7629e58496648fb33890bf25590"}
Sep 11 12:58:38.394672 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:38.394601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqgxp\" (UniqueName: \"kubernetes.io/projected/9fa02b3d-6bfc-4c28-bb1f-1f41a2693781-kube-api-access-dqgxp\") pod \"authorino-59c974bd48-t92hf\" (UID: \"9fa02b3d-6bfc-4c28-bb1f-1f41a2693781\") " pod="kuadrant-system/authorino-59c974bd48-t92hf"
Sep 11 12:58:38.404423 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:38.404347    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqgxp\" (UniqueName: \"kubernetes.io/projected/9fa02b3d-6bfc-4c28-bb1f-1f41a2693781-kube-api-access-dqgxp\") pod \"authorino-59c974bd48-t92hf\" (UID: \"9fa02b3d-6bfc-4c28-bb1f-1f41a2693781\") " pod="kuadrant-system/authorino-59c974bd48-t92hf"
Sep 11 12:58:38.460329 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:38.460293    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-t92hf"
Sep 11 12:58:38.740047 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:38.740015    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-t92hf"]
Sep 11 12:58:38.821814 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:58:38.821777    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fa02b3d_6bfc_4c28_bb1f_1f41a2693781.slice/crio-96aa23fbc7be1a1f7ae491788551e78592a6f77934b7f9e48229ac7dcebd1209 WatchSource:0}: Error finding container 96aa23fbc7be1a1f7ae491788551e78592a6f77934b7f9e48229ac7dcebd1209: Status 404 returned error can't find the container with id 96aa23fbc7be1a1f7ae491788551e78592a6f77934b7f9e48229ac7dcebd1209
Sep 11 12:58:39.380380 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:39.380343    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx" event={"ID":"0761664b-d44f-4b7d-856a-ec4c94695340","Type":"ContainerStarted","Data":"61c1a88c178861b96811353b66235ace7acdfc3cadcc0fb6eecfbdd3bc441d7a"}
Sep 11 12:58:39.380868 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:39.380542    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:39.381694 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:39.381671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-t92hf" event={"ID":"9fa02b3d-6bfc-4c28-bb1f-1f41a2693781","Type":"ContainerStarted","Data":"96aa23fbc7be1a1f7ae491788551e78592a6f77934b7f9e48229ac7dcebd1209"}
Sep 11 12:58:39.399389 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:39.399335    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx" podStartSLOduration=1.29375089 podStartE2EDuration="2.399314699s" totalImagesPullingTime="1.105563809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:58:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:58:39.397857585 +0000 UTC m=+687.460164555" watchObservedRunningTime="2026-09-11 12:58:39.399314699 +0000 UTC m=+687.461621656"
Sep 11 12:58:41.394939 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:41.394902    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-t92hf" event={"ID":"9fa02b3d-6bfc-4c28-bb1f-1f41a2693781","Type":"ContainerStarted","Data":"eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109"}
Sep 11 12:58:50.389310 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:50.389273    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:50.407200 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:50.407143    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-t92hf" podStartSLOduration=10.689313311 podStartE2EDuration="12.407126933s" totalImagesPullingTime="1.717813622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:58:41.41059038 +0000 UTC m=+689.472897338" watchObservedRunningTime="2026-09-11 12:58:50.407126933 +0000 UTC m=+698.469433880"
Sep 11 12:58:57.604989 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:57.604951    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lkcfx"]
Sep 11 12:58:57.605430 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:57.605147    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx" podUID="0761664b-d44f-4b7d-856a-ec4c94695340" containerName="limitador" containerID="cri-o://61c1a88c178861b96811353b66235ace7acdfc3cadcc0fb6eecfbdd3bc441d7a" gracePeriod=30
Sep 11 12:58:58.478980 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:58.478942    2564 generic.go:356] "Generic (PLEG): container finished" podID="0761664b-d44f-4b7d-856a-ec4c94695340" containerID="61c1a88c178861b96811353b66235ace7acdfc3cadcc0fb6eecfbdd3bc441d7a" exitCode=0
Sep 11 12:58:58.483958 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:58.483925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx" event={"ID":"0761664b-d44f-4b7d-856a-ec4c94695340","Type":"ContainerDied","Data":"61c1a88c178861b96811353b66235ace7acdfc3cadcc0fb6eecfbdd3bc441d7a"}
Sep 11 12:58:58.561922 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:58.561896    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:58.577964 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:58.577934    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcqlw\" (UniqueName: \"kubernetes.io/projected/0761664b-d44f-4b7d-856a-ec4c94695340-kube-api-access-zcqlw\") pod \"0761664b-d44f-4b7d-856a-ec4c94695340\" (UID: \"0761664b-d44f-4b7d-856a-ec4c94695340\") "
Sep 11 12:58:58.578066 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:58.578013    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0761664b-d44f-4b7d-856a-ec4c94695340-config-file\") pod \"0761664b-d44f-4b7d-856a-ec4c94695340\" (UID: \"0761664b-d44f-4b7d-856a-ec4c94695340\") "
Sep 11 12:58:58.578377 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:58.578354    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0761664b-d44f-4b7d-856a-ec4c94695340-config-file" (OuterVolumeSpecName: "config-file") pod "0761664b-d44f-4b7d-856a-ec4c94695340" (UID: "0761664b-d44f-4b7d-856a-ec4c94695340"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:58:58.580532 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:58.580498    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0761664b-d44f-4b7d-856a-ec4c94695340-kube-api-access-zcqlw" (OuterVolumeSpecName: "kube-api-access-zcqlw") pod "0761664b-d44f-4b7d-856a-ec4c94695340" (UID: "0761664b-d44f-4b7d-856a-ec4c94695340"). InnerVolumeSpecName "kube-api-access-zcqlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:58:58.679552 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:58.679445    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcqlw\" (UniqueName: \"kubernetes.io/projected/0761664b-d44f-4b7d-856a-ec4c94695340-kube-api-access-zcqlw\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:58:58.679552 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:58.679492    2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0761664b-d44f-4b7d-856a-ec4c94695340-config-file\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:58:59.485774 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.485745    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx"
Sep 11 12:58:59.485951 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.485775    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lkcfx" event={"ID":"0761664b-d44f-4b7d-856a-ec4c94695340","Type":"ContainerDied","Data":"49f26287d732c0d1fd1efeaa78eb1667e8c3b7629e58496648fb33890bf25590"}
Sep 11 12:58:59.485951 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.485837    2564 scope.go:117] "RemoveContainer" containerID="61c1a88c178861b96811353b66235ace7acdfc3cadcc0fb6eecfbdd3bc441d7a"
Sep 11 12:58:59.507826 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.507786    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lkcfx"]
Sep 11 12:58:59.514607 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.514504    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lkcfx"]
Sep 11 12:58:59.790043 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.789950    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"]
Sep 11 12:58:59.790529 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.790514    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0761664b-d44f-4b7d-856a-ec4c94695340" containerName="limitador"
Sep 11 12:58:59.790571 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.790533    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0761664b-d44f-4b7d-856a-ec4c94695340" containerName="limitador"
Sep 11 12:58:59.790628 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.790617    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0761664b-d44f-4b7d-856a-ec4c94695340" containerName="limitador"
Sep 11 12:58:59.796488 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.796459    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"
Sep 11 12:58:59.799654 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.799617    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-b7hss\""
Sep 11 12:58:59.799654 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.799621    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 12:58:59.799872 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.799757    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 12:58:59.800396 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.800379    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 12:58:59.804598 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.804570    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"]
Sep 11 12:58:59.892206 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.892159    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx8hm\" (UniqueName: \"kubernetes.io/projected/38ec5f72-e3b3-4e64-a818-57fc316658b4-kube-api-access-lx8hm\") pod \"postgres-69fb4594cf-jpqmg\" (UID: \"38ec5f72-e3b3-4e64-a818-57fc316658b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"
Sep 11 12:58:59.892206 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.892205    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/38ec5f72-e3b3-4e64-a818-57fc316658b4-data\") pod \"postgres-69fb4594cf-jpqmg\" (UID: \"38ec5f72-e3b3-4e64-a818-57fc316658b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"
Sep 11 12:58:59.993686 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.993621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx8hm\" (UniqueName: \"kubernetes.io/projected/38ec5f72-e3b3-4e64-a818-57fc316658b4-kube-api-access-lx8hm\") pod \"postgres-69fb4594cf-jpqmg\" (UID: \"38ec5f72-e3b3-4e64-a818-57fc316658b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"
Sep 11 12:58:59.993914 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.993765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/38ec5f72-e3b3-4e64-a818-57fc316658b4-data\") pod \"postgres-69fb4594cf-jpqmg\" (UID: \"38ec5f72-e3b3-4e64-a818-57fc316658b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"
Sep 11 12:58:59.994160 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:58:59.994136    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/38ec5f72-e3b3-4e64-a818-57fc316658b4-data\") pod \"postgres-69fb4594cf-jpqmg\" (UID: \"38ec5f72-e3b3-4e64-a818-57fc316658b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"
Sep 11 12:59:00.002242 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:00.002211    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx8hm\" (UniqueName: \"kubernetes.io/projected/38ec5f72-e3b3-4e64-a818-57fc316658b4-kube-api-access-lx8hm\") pod \"postgres-69fb4594cf-jpqmg\" (UID: \"38ec5f72-e3b3-4e64-a818-57fc316658b4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"
Sep 11 12:59:00.109759 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:00.109700    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"
Sep 11 12:59:00.383237 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:00.383213    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"]
Sep 11 12:59:00.385277 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:59:00.385253    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38ec5f72_e3b3_4e64_a818_57fc316658b4.slice/crio-97d246dc654d17393fc68a49ded55aae7940af2958f192d90e638eea93eee939 WatchSource:0}: Error finding container 97d246dc654d17393fc68a49ded55aae7940af2958f192d90e638eea93eee939: Status 404 returned error can't find the container with id 97d246dc654d17393fc68a49ded55aae7940af2958f192d90e638eea93eee939
Sep 11 12:59:00.483456 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:00.483412    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0761664b-d44f-4b7d-856a-ec4c94695340" path="/var/lib/kubelet/pods/0761664b-d44f-4b7d-856a-ec4c94695340/volumes"
Sep 11 12:59:00.491828 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:00.491793    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg" event={"ID":"38ec5f72-e3b3-4e64-a818-57fc316658b4","Type":"ContainerStarted","Data":"97d246dc654d17393fc68a49ded55aae7940af2958f192d90e638eea93eee939"}
Sep 11 12:59:07.535367 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:07.535324    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg" event={"ID":"38ec5f72-e3b3-4e64-a818-57fc316658b4","Type":"ContainerStarted","Data":"e4a068d72f5e7dea2abf38927143f5ad964af9e14f59bad0ac1cac111d75989a"}
Sep 11 12:59:07.535775 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:07.535441    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"
Sep 11 12:59:07.554123 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:07.554065    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg" podStartSLOduration=1.5700091189999998 podStartE2EDuration="8.554047522s" totalImagesPullingTime="6.984038403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:59:07.551990504 +0000 UTC m=+715.614297465" watchObservedRunningTime="2026-09-11 12:59:07.554047522 +0000 UTC m=+715.616354479"
Sep 11 12:59:13.568262 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:13.568229    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpqmg"
Sep 11 12:59:14.402596 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.402536    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66d75f6c7-svpz4"]
Sep 11 12:59:14.407342 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.407315    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d75f6c7-svpz4"
Sep 11 12:59:14.409815 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.409791    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 12:59:14.415080 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.415049    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66d75f6c7-svpz4"]
Sep 11 12:59:14.515876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.515836    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b4034f90-8419-46e0-b892-da93843a3bc0-tls-cert\") pod \"authorino-66d75f6c7-svpz4\" (UID: \"b4034f90-8419-46e0-b892-da93843a3bc0\") " pod="kuadrant-system/authorino-66d75f6c7-svpz4"
Sep 11 12:59:14.516050 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.515909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjjj7\" (UniqueName: \"kubernetes.io/projected/b4034f90-8419-46e0-b892-da93843a3bc0-kube-api-access-rjjj7\") pod \"authorino-66d75f6c7-svpz4\" (UID: \"b4034f90-8419-46e0-b892-da93843a3bc0\") " pod="kuadrant-system/authorino-66d75f6c7-svpz4"
Sep 11 12:59:14.616535 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.616494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b4034f90-8419-46e0-b892-da93843a3bc0-tls-cert\") pod \"authorino-66d75f6c7-svpz4\" (UID: \"b4034f90-8419-46e0-b892-da93843a3bc0\") " pod="kuadrant-system/authorino-66d75f6c7-svpz4"
Sep 11 12:59:14.616535 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.616540    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjjj7\" (UniqueName: \"kubernetes.io/projected/b4034f90-8419-46e0-b892-da93843a3bc0-kube-api-access-rjjj7\") pod \"authorino-66d75f6c7-svpz4\" (UID: \"b4034f90-8419-46e0-b892-da93843a3bc0\") " pod="kuadrant-system/authorino-66d75f6c7-svpz4"
Sep 11 12:59:14.618986 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.618965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b4034f90-8419-46e0-b892-da93843a3bc0-tls-cert\") pod \"authorino-66d75f6c7-svpz4\" (UID: \"b4034f90-8419-46e0-b892-da93843a3bc0\") " pod="kuadrant-system/authorino-66d75f6c7-svpz4"
Sep 11 12:59:14.627376 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.627339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjjj7\" (UniqueName: \"kubernetes.io/projected/b4034f90-8419-46e0-b892-da93843a3bc0-kube-api-access-rjjj7\") pod \"authorino-66d75f6c7-svpz4\" (UID: \"b4034f90-8419-46e0-b892-da93843a3bc0\") " pod="kuadrant-system/authorino-66d75f6c7-svpz4"
Sep 11 12:59:14.719310 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.719220    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d75f6c7-svpz4"
Sep 11 12:59:14.990132 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:14.990105    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66d75f6c7-svpz4"]
Sep 11 12:59:14.992946 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:59:14.992918    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4034f90_8419_46e0_b892_da93843a3bc0.slice/crio-68788bd75f9014853b812eaaebb50f99f230ac6c5815dc3411779a4bcfd4b46f WatchSource:0}: Error finding container 68788bd75f9014853b812eaaebb50f99f230ac6c5815dc3411779a4bcfd4b46f: Status 404 returned error can't find the container with id 68788bd75f9014853b812eaaebb50f99f230ac6c5815dc3411779a4bcfd4b46f
Sep 11 12:59:15.575212 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:15.575167    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d75f6c7-svpz4" event={"ID":"b4034f90-8419-46e0-b892-da93843a3bc0","Type":"ContainerStarted","Data":"68788bd75f9014853b812eaaebb50f99f230ac6c5815dc3411779a4bcfd4b46f"}
Sep 11 12:59:16.581133 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:16.581099    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d75f6c7-svpz4" event={"ID":"b4034f90-8419-46e0-b892-da93843a3bc0","Type":"ContainerStarted","Data":"aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5"}
Sep 11 12:59:16.598291 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:16.598233    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66d75f6c7-svpz4" podStartSLOduration=1.54935855 podStartE2EDuration="2.598213741s" totalImagesPullingTime="1.048855191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:59:16.597696679 +0000 UTC m=+724.660003636" watchObservedRunningTime="2026-09-11 12:59:16.598213741 +0000 UTC m=+724.660520702"
Sep 11 12:59:16.626604 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:16.626557    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-t92hf"]
Sep 11 12:59:16.626831 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:16.626800    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-t92hf" podUID="9fa02b3d-6bfc-4c28-bb1f-1f41a2693781" containerName="authorino" containerID="cri-o://eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109" gracePeriod=30
Sep 11 12:59:16.877807 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:16.877776    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-t92hf"
Sep 11 12:59:16.938731 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:16.938691    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqgxp\" (UniqueName: \"kubernetes.io/projected/9fa02b3d-6bfc-4c28-bb1f-1f41a2693781-kube-api-access-dqgxp\") pod \"9fa02b3d-6bfc-4c28-bb1f-1f41a2693781\" (UID: \"9fa02b3d-6bfc-4c28-bb1f-1f41a2693781\") "
Sep 11 12:59:16.940816 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:16.940789    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa02b3d-6bfc-4c28-bb1f-1f41a2693781-kube-api-access-dqgxp" (OuterVolumeSpecName: "kube-api-access-dqgxp") pod "9fa02b3d-6bfc-4c28-bb1f-1f41a2693781" (UID: "9fa02b3d-6bfc-4c28-bb1f-1f41a2693781"). InnerVolumeSpecName "kube-api-access-dqgxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:59:17.040275 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:17.040235    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqgxp\" (UniqueName: \"kubernetes.io/projected/9fa02b3d-6bfc-4c28-bb1f-1f41a2693781-kube-api-access-dqgxp\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:59:17.586685 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:17.586624    2564 generic.go:356] "Generic (PLEG): container finished" podID="9fa02b3d-6bfc-4c28-bb1f-1f41a2693781" containerID="eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109" exitCode=0
Sep 11 12:59:17.587103 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:17.586688    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-t92hf"
Sep 11 12:59:17.587103 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:17.586721    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-t92hf" event={"ID":"9fa02b3d-6bfc-4c28-bb1f-1f41a2693781","Type":"ContainerDied","Data":"eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109"}
Sep 11 12:59:17.587103 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:17.586764    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-t92hf" event={"ID":"9fa02b3d-6bfc-4c28-bb1f-1f41a2693781","Type":"ContainerDied","Data":"96aa23fbc7be1a1f7ae491788551e78592a6f77934b7f9e48229ac7dcebd1209"}
Sep 11 12:59:17.587103 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:17.586787    2564 scope.go:117] "RemoveContainer" containerID="eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109"
Sep 11 12:59:17.595803 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:17.595783    2564 scope.go:117] "RemoveContainer" containerID="eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109"
Sep 11 12:59:17.596114 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:59:17.596093    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109\": container with ID starting with eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109 not found: ID does not exist" containerID="eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109"
Sep 11 12:59:17.596163 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:17.596122    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109"} err="failed to get container status \"eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109\": rpc error: code = NotFound desc = could not find container \"eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109\": container with ID starting with eb443f9c4eccbd38c381431b4e89c0099057637f67146b21411a5c214d06b109 not found: ID does not exist"
Sep 11 12:59:17.608368 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:17.608337    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-t92hf"]
Sep 11 12:59:17.612393 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:17.612367    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-t92hf"]
Sep 11 12:59:18.482949 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:18.482915    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fa02b3d-6bfc-4c28-bb1f-1f41a2693781" path="/var/lib/kubelet/pods/9fa02b3d-6bfc-4c28-bb1f-1f41a2693781/volumes"
Sep 11 12:59:21.658408 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.658365    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d9bd4c4c-g4mwf"]
Sep 11 12:59:21.658830 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.658779    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa02b3d-6bfc-4c28-bb1f-1f41a2693781" containerName="authorino"
Sep 11 12:59:21.658830 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.658795    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa02b3d-6bfc-4c28-bb1f-1f41a2693781" containerName="authorino"
Sep 11 12:59:21.658901 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.658865    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fa02b3d-6bfc-4c28-bb1f-1f41a2693781" containerName="authorino"
Sep 11 12:59:21.663370 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.663343    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:21.667619 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.667593    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 12:59:21.667918 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.667900    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 12:59:21.668036 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.668016    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 12:59:21.668716 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.668701    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kxqtj\""
Sep 11 12:59:21.676078 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.676038    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d9bd4c4c-g4mwf"]
Sep 11 12:59:21.786567 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.786527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-webhook-cert\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:21.786753 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.786574    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hlnp\" (UniqueName: \"kubernetes.io/projected/28fc1900-92da-4de5-b78c-1d35f698f983-kube-api-access-2hlnp\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:21.786753 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.786690    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-metrics-certs\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:21.858858 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.858822    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-758b8949b9-2qctg"]
Sep 11 12:59:21.862172 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.862148    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:21.873983 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.873953    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-758b8949b9-2qctg"]
Sep 11 12:59:21.887202 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.887169    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-metrics-certs\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:21.887403 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.887306    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-webhook-cert\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:21.887403 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:59:21.887363    2564 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 11 12:59:21.887503 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:59:21.887434    2564 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 11 12:59:21.887503 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:59:21.887435    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-metrics-certs podName:28fc1900-92da-4de5-b78c-1d35f698f983 nodeName:}" failed. No retries permitted until 2026-09-11 12:59:22.387413573 +0000 UTC m=+730.449720518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-metrics-certs") pod "maas-controller-5d9bd4c4c-g4mwf" (UID: "28fc1900-92da-4de5-b78c-1d35f698f983") : secret "maas-controller-metrics-tls" not found
Sep 11 12:59:21.887610 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:59:21.887507    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-webhook-cert podName:28fc1900-92da-4de5-b78c-1d35f698f983 nodeName:}" failed. No retries permitted until 2026-09-11 12:59:22.387488889 +0000 UTC m=+730.449795831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-webhook-cert") pod "maas-controller-5d9bd4c4c-g4mwf" (UID: "28fc1900-92da-4de5-b78c-1d35f698f983") : secret "maas-controller-webhook-cert" not found
Sep 11 12:59:21.887610 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.887524    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hlnp\" (UniqueName: \"kubernetes.io/projected/28fc1900-92da-4de5-b78c-1d35f698f983-kube-api-access-2hlnp\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:21.898559 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.898510    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hlnp\" (UniqueName: \"kubernetes.io/projected/28fc1900-92da-4de5-b78c-1d35f698f983-kube-api-access-2hlnp\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:21.988328 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.988244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-webhook-cert\") pod \"maas-controller-758b8949b9-2qctg\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") " pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:21.988328 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.988294    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-metrics-certs\") pod \"maas-controller-758b8949b9-2qctg\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") " pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:21.988328 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:21.988313    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdfhz\" (UniqueName: \"kubernetes.io/projected/bb3a5593-6320-4701-b33f-93ecd7aea9fe-kube-api-access-sdfhz\") pod \"maas-controller-758b8949b9-2qctg\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") " pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:22.089726 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.089683    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-webhook-cert\") pod \"maas-controller-758b8949b9-2qctg\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") " pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:22.089928 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.089749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-metrics-certs\") pod \"maas-controller-758b8949b9-2qctg\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") " pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:22.089928 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.089771    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdfhz\" (UniqueName: \"kubernetes.io/projected/bb3a5593-6320-4701-b33f-93ecd7aea9fe-kube-api-access-sdfhz\") pod \"maas-controller-758b8949b9-2qctg\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") " pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:22.092174 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.092146    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-metrics-certs\") pod \"maas-controller-758b8949b9-2qctg\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") " pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:22.092290 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.092175    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-webhook-cert\") pod \"maas-controller-758b8949b9-2qctg\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") " pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:22.098777 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.098755    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdfhz\" (UniqueName: \"kubernetes.io/projected/bb3a5593-6320-4701-b33f-93ecd7aea9fe-kube-api-access-sdfhz\") pod \"maas-controller-758b8949b9-2qctg\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") " pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:22.172600 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.172555    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:22.392424 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.392387    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-metrics-certs\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:22.392627 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.392454    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-webhook-cert\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:22.395054 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.395022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-metrics-certs\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:22.395156 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.395022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-webhook-cert\") pod \"maas-controller-5d9bd4c4c-g4mwf\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") " pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:22.444862 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.444834    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-758b8949b9-2qctg"]
Sep 11 12:59:22.446354 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:59:22.446332    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3a5593_6320_4701_b33f_93ecd7aea9fe.slice/crio-02da4ba6d0e461c4ec7e5d38ac1bfda2fb2792cf081b9430594571a36168cfe0 WatchSource:0}: Error finding container 02da4ba6d0e461c4ec7e5d38ac1bfda2fb2792cf081b9430594571a36168cfe0: Status 404 returned error can't find the container with id 02da4ba6d0e461c4ec7e5d38ac1bfda2fb2792cf081b9430594571a36168cfe0
Sep 11 12:59:22.574333 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.574297    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:22.613624 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.613582    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758b8949b9-2qctg" event={"ID":"bb3a5593-6320-4701-b33f-93ecd7aea9fe","Type":"ContainerStarted","Data":"02da4ba6d0e461c4ec7e5d38ac1bfda2fb2792cf081b9430594571a36168cfe0"}
Sep 11 12:59:22.851086 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:22.851056    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d9bd4c4c-g4mwf"]
Sep 11 12:59:22.854223 ip-10-0-136-55 kubenswrapper[2564]: W0911 12:59:22.854193    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28fc1900_92da_4de5_b78c_1d35f698f983.slice/crio-5a879168aecbd1663bef9f0c12f1cc27f04ecf1501a501380777b6d07a58d802 WatchSource:0}: Error finding container 5a879168aecbd1663bef9f0c12f1cc27f04ecf1501a501380777b6d07a58d802: Status 404 returned error can't find the container with id 5a879168aecbd1663bef9f0c12f1cc27f04ecf1501a501380777b6d07a58d802
Sep 11 12:59:23.623983 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:23.623944    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf" event={"ID":"28fc1900-92da-4de5-b78c-1d35f698f983","Type":"ContainerStarted","Data":"5a879168aecbd1663bef9f0c12f1cc27f04ecf1501a501380777b6d07a58d802"}
Sep 11 12:59:25.638546 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:25.638510    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf" event={"ID":"28fc1900-92da-4de5-b78c-1d35f698f983","Type":"ContainerStarted","Data":"70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00"}
Sep 11 12:59:25.639035 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:25.638577    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:25.640144 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:25.640108    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758b8949b9-2qctg" event={"ID":"bb3a5593-6320-4701-b33f-93ecd7aea9fe","Type":"ContainerStarted","Data":"4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0"}
Sep 11 12:59:25.640258 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:25.640245    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:25.661288 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:25.661234    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf" podStartSLOduration=2.349534407 podStartE2EDuration="4.661217556s" totalImagesPullingTime="2.311683149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:59:25.659610093 +0000 UTC m=+733.721917053" watchObservedRunningTime="2026-09-11 12:59:25.661217556 +0000 UTC m=+733.723524513"
Sep 11 12:59:36.648507 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:36.648471    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 12:59:36.649017 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:36.648559    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:36.680460 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:36.680395    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-758b8949b9-2qctg" podStartSLOduration=12.967388366 podStartE2EDuration="15.680376783s" totalImagesPullingTime="2.712988417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:59:25.678629627 +0000 UTC m=+733.740936585" watchObservedRunningTime="2026-09-11 12:59:36.680376783 +0000 UTC m=+744.742683742"
Sep 11 12:59:36.760770 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:36.760731    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d9bd4c4c-g4mwf"]
Sep 11 12:59:36.760991 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:36.760965    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf" podUID="28fc1900-92da-4de5-b78c-1d35f698f983" containerName="manager" containerID="cri-o://70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00" gracePeriod=30
Sep 11 12:59:37.015876 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.015843    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:37.131931 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.131895    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hlnp\" (UniqueName: \"kubernetes.io/projected/28fc1900-92da-4de5-b78c-1d35f698f983-kube-api-access-2hlnp\") pod \"28fc1900-92da-4de5-b78c-1d35f698f983\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") "
Sep 11 12:59:37.132112 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.132098    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-metrics-certs\") pod \"28fc1900-92da-4de5-b78c-1d35f698f983\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") "
Sep 11 12:59:37.132152 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.132127    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-webhook-cert\") pod \"28fc1900-92da-4de5-b78c-1d35f698f983\" (UID: \"28fc1900-92da-4de5-b78c-1d35f698f983\") "
Sep 11 12:59:37.134189 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.134149    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28fc1900-92da-4de5-b78c-1d35f698f983-kube-api-access-2hlnp" (OuterVolumeSpecName: "kube-api-access-2hlnp") pod "28fc1900-92da-4de5-b78c-1d35f698f983" (UID: "28fc1900-92da-4de5-b78c-1d35f698f983"). InnerVolumeSpecName "kube-api-access-2hlnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:59:37.134336 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.134228    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "28fc1900-92da-4de5-b78c-1d35f698f983" (UID: "28fc1900-92da-4de5-b78c-1d35f698f983"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:59:37.134336 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.134251    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "28fc1900-92da-4de5-b78c-1d35f698f983" (UID: "28fc1900-92da-4de5-b78c-1d35f698f983"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:59:37.233452 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.233334    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:59:37.233452 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.233387    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fc1900-92da-4de5-b78c-1d35f698f983-webhook-cert\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:59:37.233452 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.233400    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hlnp\" (UniqueName: \"kubernetes.io/projected/28fc1900-92da-4de5-b78c-1d35f698f983-kube-api-access-2hlnp\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 12:59:37.705006 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.704970    2564 generic.go:356] "Generic (PLEG): container finished" podID="28fc1900-92da-4de5-b78c-1d35f698f983" containerID="70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00" exitCode=0
Sep 11 12:59:37.705449 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.705015    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf"
Sep 11 12:59:37.705449 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.705049    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf" event={"ID":"28fc1900-92da-4de5-b78c-1d35f698f983","Type":"ContainerDied","Data":"70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00"}
Sep 11 12:59:37.705449 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.705102    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d9bd4c4c-g4mwf" event={"ID":"28fc1900-92da-4de5-b78c-1d35f698f983","Type":"ContainerDied","Data":"5a879168aecbd1663bef9f0c12f1cc27f04ecf1501a501380777b6d07a58d802"}
Sep 11 12:59:37.705449 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.705126    2564 scope.go:117] "RemoveContainer" containerID="70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00"
Sep 11 12:59:37.714926 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.714903    2564 scope.go:117] "RemoveContainer" containerID="70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00"
Sep 11 12:59:37.715266 ip-10-0-136-55 kubenswrapper[2564]: E0911 12:59:37.715247    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00\": container with ID starting with 70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00 not found: ID does not exist" containerID="70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00"
Sep 11 12:59:37.715323 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.715276    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00"} err="failed to get container status \"70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00\": rpc error: code = NotFound desc = could not find container \"70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00\": container with ID starting with 70c372e3bb1ac27e39f80f9c11728f25555598f43cd508f277393976ab6a7d00 not found: ID does not exist"
Sep 11 12:59:37.728853 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.728816    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d9bd4c4c-g4mwf"]
Sep 11 12:59:37.735588 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:37.735555    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5d9bd4c4c-g4mwf"]
Sep 11 12:59:38.483770 ip-10-0-136-55 kubenswrapper[2564]: I0911 12:59:38.483736    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28fc1900-92da-4de5-b78c-1d35f698f983" path="/var/lib/kubelet/pods/28fc1900-92da-4de5-b78c-1d35f698f983/volumes"
Sep 11 13:00:00.666946 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.666908    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"]
Sep 11 13:00:00.667412 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.667268    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28fc1900-92da-4de5-b78c-1d35f698f983" containerName="manager"
Sep 11 13:00:00.667412 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.667279    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="28fc1900-92da-4de5-b78c-1d35f698f983" containerName="manager"
Sep 11 13:00:00.667412 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.667347    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="28fc1900-92da-4de5-b78c-1d35f698f983" containerName="manager"
Sep 11 13:00:00.692445 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.692414    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"]
Sep 11 13:00:00.692602 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.692535    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.695284 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.695260    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 13:00:00.695440 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.695262    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 13:00:00.695440 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.695392    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2t5gt\""
Sep 11 13:00:00.728496 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.728458    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpl2p\" (UniqueName: \"kubernetes.io/projected/aa05ff05-d220-4d14-9374-ab47e05f5b6c-kube-api-access-zpl2p\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.728693 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.728506    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-metrics-certs\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.728693 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.728568    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa05ff05-d220-4d14-9374-ab47e05f5b6c-openshift-service-ca\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.728693 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.728604    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-maas-api-tls\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.830308 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.830275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-metrics-certs\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.830308 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.830315    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa05ff05-d220-4d14-9374-ab47e05f5b6c-openshift-service-ca\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.830542 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.830335    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-maas-api-tls\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.830542 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.830425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpl2p\" (UniqueName: \"kubernetes.io/projected/aa05ff05-d220-4d14-9374-ab47e05f5b6c-kube-api-access-zpl2p\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.831071 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.831042    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa05ff05-d220-4d14-9374-ab47e05f5b6c-openshift-service-ca\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.832860 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.832832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-maas-api-tls\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.832950 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.832903    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-metrics-certs\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:00.838618 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:00.838592    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpl2p\" (UniqueName: \"kubernetes.io/projected/aa05ff05-d220-4d14-9374-ab47e05f5b6c-kube-api-access-zpl2p\") pod \"maas-api-7ff8fbcd4f-tjwrn\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") " pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:01.004281 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.004197    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:01.283101 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.283072    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"]
Sep 11 13:00:01.285430 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:00:01.285402    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa05ff05_d220_4d14_9374_ab47e05f5b6c.slice/crio-25e97d5923550514bbaebf4f359e4ab94fc109cd39b12fa97dddabab318559bf WatchSource:0}: Error finding container 25e97d5923550514bbaebf4f359e4ab94fc109cd39b12fa97dddabab318559bf: Status 404 returned error can't find the container with id 25e97d5923550514bbaebf4f359e4ab94fc109cd39b12fa97dddabab318559bf
Sep 11 13:00:01.372914 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.372875    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"]
Sep 11 13:00:01.377678 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.377650    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"
Sep 11 13:00:01.380722 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.380692    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jws2n\""
Sep 11 13:00:01.380864 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.380725    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 13:00:01.391097 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.391068    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"]
Sep 11 13:00:01.435406 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.435368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6mwq\" (UniqueName: \"kubernetes.io/projected/82825f9e-f8bd-4fcc-b301-eaa5031c3f6e-kube-api-access-f6mwq\") pod \"payload-processing-5b54b7fbb7-dsj2b\" (UID: \"82825f9e-f8bd-4fcc-b301-eaa5031c3f6e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"
Sep 11 13:00:01.435406 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.435408    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82825f9e-f8bd-4fcc-b301-eaa5031c3f6e-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-dsj2b\" (UID: \"82825f9e-f8bd-4fcc-b301-eaa5031c3f6e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"
Sep 11 13:00:01.437266 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.437243    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"]
Sep 11 13:00:01.440739 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.440719    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"
Sep 11 13:00:01.468785 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.468746    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"]
Sep 11 13:00:01.536285 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.536182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82825f9e-f8bd-4fcc-b301-eaa5031c3f6e-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-dsj2b\" (UID: \"82825f9e-f8bd-4fcc-b301-eaa5031c3f6e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"
Sep 11 13:00:01.536285 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.536237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bfac1c8-4103-480f-aad9-afa5be62df45-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-rhvp5\" (UID: \"5bfac1c8-4103-480f-aad9-afa5be62df45\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"
Sep 11 13:00:01.536525 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.536386    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6mwq\" (UniqueName: \"kubernetes.io/projected/82825f9e-f8bd-4fcc-b301-eaa5031c3f6e-kube-api-access-f6mwq\") pod \"payload-processing-5b54b7fbb7-dsj2b\" (UID: \"82825f9e-f8bd-4fcc-b301-eaa5031c3f6e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"
Sep 11 13:00:01.536525 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.536413    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tnv4\" (UniqueName: \"kubernetes.io/projected/5bfac1c8-4103-480f-aad9-afa5be62df45-kube-api-access-8tnv4\") pod \"payload-pre-processing-59fbc47bf8-rhvp5\" (UID: \"5bfac1c8-4103-480f-aad9-afa5be62df45\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"
Sep 11 13:00:01.537499 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.537463    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82825f9e-f8bd-4fcc-b301-eaa5031c3f6e-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-dsj2b\" (UID: \"82825f9e-f8bd-4fcc-b301-eaa5031c3f6e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"
Sep 11 13:00:01.549002 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.548971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6mwq\" (UniqueName: \"kubernetes.io/projected/82825f9e-f8bd-4fcc-b301-eaa5031c3f6e-kube-api-access-f6mwq\") pod \"payload-processing-5b54b7fbb7-dsj2b\" (UID: \"82825f9e-f8bd-4fcc-b301-eaa5031c3f6e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"
Sep 11 13:00:01.637206 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.637098    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tnv4\" (UniqueName: \"kubernetes.io/projected/5bfac1c8-4103-480f-aad9-afa5be62df45-kube-api-access-8tnv4\") pod \"payload-pre-processing-59fbc47bf8-rhvp5\" (UID: \"5bfac1c8-4103-480f-aad9-afa5be62df45\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"
Sep 11 13:00:01.637407 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.637249    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bfac1c8-4103-480f-aad9-afa5be62df45-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-rhvp5\" (UID: \"5bfac1c8-4103-480f-aad9-afa5be62df45\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"
Sep 11 13:00:01.638253 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.638226    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bfac1c8-4103-480f-aad9-afa5be62df45-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-rhvp5\" (UID: \"5bfac1c8-4103-480f-aad9-afa5be62df45\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"
Sep 11 13:00:01.646799 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.646763    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tnv4\" (UniqueName: \"kubernetes.io/projected/5bfac1c8-4103-480f-aad9-afa5be62df45-kube-api-access-8tnv4\") pod \"payload-pre-processing-59fbc47bf8-rhvp5\" (UID: \"5bfac1c8-4103-480f-aad9-afa5be62df45\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"
Sep 11 13:00:01.688973 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.688933    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"
Sep 11 13:00:01.751525 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.750607    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"
Sep 11 13:00:01.828660 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.828533    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn" event={"ID":"aa05ff05-d220-4d14-9374-ab47e05f5b6c","Type":"ContainerStarted","Data":"25e97d5923550514bbaebf4f359e4ab94fc109cd39b12fa97dddabab318559bf"}
Sep 11 13:00:01.973541 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:01.973493    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"]
Sep 11 13:00:01.975103 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:00:01.975062    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82825f9e_f8bd_4fcc_b301_eaa5031c3f6e.slice/crio-de2606ab150fe4512ffccd84ab4784fab2caea37cb965a2ad33d83b6552ddb58 WatchSource:0}: Error finding container de2606ab150fe4512ffccd84ab4784fab2caea37cb965a2ad33d83b6552ddb58: Status 404 returned error can't find the container with id de2606ab150fe4512ffccd84ab4784fab2caea37cb965a2ad33d83b6552ddb58
Sep 11 13:00:02.041282 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:02.041218    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"]
Sep 11 13:00:02.044491 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:00:02.044458    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfac1c8_4103_480f_aad9_afa5be62df45.slice/crio-2963e640c9885d6930b22e758b62f92ae02e001b46b9fd79bdb3e71d18033a7d WatchSource:0}: Error finding container 2963e640c9885d6930b22e758b62f92ae02e001b46b9fd79bdb3e71d18033a7d: Status 404 returned error can't find the container with id 2963e640c9885d6930b22e758b62f92ae02e001b46b9fd79bdb3e71d18033a7d
Sep 11 13:00:02.835197 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:02.835156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5" event={"ID":"5bfac1c8-4103-480f-aad9-afa5be62df45","Type":"ContainerStarted","Data":"2963e640c9885d6930b22e758b62f92ae02e001b46b9fd79bdb3e71d18033a7d"}
Sep 11 13:00:02.836654 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:02.836612    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b" event={"ID":"82825f9e-f8bd-4fcc-b301-eaa5031c3f6e","Type":"ContainerStarted","Data":"de2606ab150fe4512ffccd84ab4784fab2caea37cb965a2ad33d83b6552ddb58"}
Sep 11 13:00:05.857350 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:05.857308    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5" event={"ID":"5bfac1c8-4103-480f-aad9-afa5be62df45","Type":"ContainerStarted","Data":"3bb0f1c69606f95d0ab0ea986e7e2c4a025c587f0294397c79382cffeb4e36a1"}
Sep 11 13:00:05.857805 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:05.857379    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"
Sep 11 13:00:05.858981 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:05.858953    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b" event={"ID":"82825f9e-f8bd-4fcc-b301-eaa5031c3f6e","Type":"ContainerStarted","Data":"731c33b5539b27032de49820e3d2bbcfa6170211f8a2f3abe4529faf96ddb7a0"}
Sep 11 13:00:05.859111 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:05.859008    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"
Sep 11 13:00:05.860500 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:05.860477    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn" event={"ID":"aa05ff05-d220-4d14-9374-ab47e05f5b6c","Type":"ContainerStarted","Data":"aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b"}
Sep 11 13:00:05.860611 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:05.860590    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:05.881666 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:05.878170    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5" podStartSLOduration=1.96843848 podStartE2EDuration="4.878146197s" totalImagesPullingTime="2.909707717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:00:05.875929147 +0000 UTC m=+773.938236120" watchObservedRunningTime="2026-09-11 13:00:05.878146197 +0000 UTC m=+773.940453160"
Sep 11 13:00:05.900226 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:05.900170    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b" podStartSLOduration=1.9279648900000002 podStartE2EDuration="4.900153232s" totalImagesPullingTime="2.972188342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:00:05.899041576 +0000 UTC m=+773.961348580" watchObservedRunningTime="2026-09-11 13:00:05.900153232 +0000 UTC m=+773.962460188"
Sep 11 13:00:05.918703 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:05.918632    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn" podStartSLOduration=2.255400288 podStartE2EDuration="5.918612824s" totalImagesPullingTime="3.663212536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:00:05.916727509 +0000 UTC m=+773.979034468" watchObservedRunningTime="2026-09-11 13:00:05.918612824 +0000 UTC m=+773.980919779"
Sep 11 13:00:11.871079 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:11.871046    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:00:16.867406 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:16.867372    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-rhvp5"
Sep 11 13:00:16.867886 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:16.867450    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-dsj2b"
Sep 11 13:00:56.604981 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.604943    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"]
Sep 11 13:00:56.615084 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.615044    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.617881 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.617851    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 13:00:56.619036 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.618955    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gqvhs\""
Sep 11 13:00:56.619356 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.618983    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 13:00:56.619356 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.619133    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 13:00:56.622080 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.622052    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"]
Sep 11 13:00:56.747849 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.747814    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.748036 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.747880    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.748036 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.747989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.748036 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.748021    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.748150 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.748041    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vc85\" (UniqueName: \"kubernetes.io/projected/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-kube-api-access-8vc85\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.748150 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.748124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.848612 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.848564    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.848836 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.848633    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.848836 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.848697    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.848836 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.848747    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.848836 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.848772    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.848836 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.848793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vc85\" (UniqueName: \"kubernetes.io/projected/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-kube-api-access-8vc85\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.849099 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.849066    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.849224 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.849078    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.849224 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.849170    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.851186 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.851155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.851476 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.851454    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.858010 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.857937    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vc85\" (UniqueName: \"kubernetes.io/projected/f883abaa-2ee4-443d-be64-0cdb2bb7c58f-kube-api-access-8vc85\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pc5w5\" (UID: \"f883abaa-2ee4-443d-be64-0cdb2bb7c58f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:56.928072 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:56.928027    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:00:57.204599 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:57.204568    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"]
Sep 11 13:00:57.206809 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:00:57.206778    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf883abaa_2ee4_443d_be64_0cdb2bb7c58f.slice/crio-15d54984a549336449e2a4738cafad12598eebbb1048fbc26ce92f4a30a95a10 WatchSource:0}: Error finding container 15d54984a549336449e2a4738cafad12598eebbb1048fbc26ce92f4a30a95a10: Status 404 returned error can't find the container with id 15d54984a549336449e2a4738cafad12598eebbb1048fbc26ce92f4a30a95a10
Sep 11 13:00:58.147164 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:58.147128    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5" event={"ID":"f883abaa-2ee4-443d-be64-0cdb2bb7c58f","Type":"ContainerStarted","Data":"15d54984a549336449e2a4738cafad12598eebbb1048fbc26ce92f4a30a95a10"}
Sep 11 13:00:59.154607 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:59.154565    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5" event={"ID":"f883abaa-2ee4-443d-be64-0cdb2bb7c58f","Type":"ContainerStarted","Data":"37a75022710f49d221a57d063cf2055c0133b4ad50e29f5940ee9a4e69334471"}
Sep 11 13:00:59.174867 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:00:59.174795    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5" podStartSLOduration=2.1843099759999998 podStartE2EDuration="3.174772689s" totalImagesPullingTime="990.462713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:00:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:00:59.173478075 +0000 UTC m=+827.235785048" watchObservedRunningTime="2026-09-11 13:00:59.174772689 +0000 UTC m=+827.237079647"
Sep 11 13:01:03.242333 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.242297    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"]
Sep 11 13:01:03.245977 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.245957    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.249138 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.249116    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 13:01:03.274847 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.274812    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"]
Sep 11 13:01:03.413270 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.413230    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/523a084f-cd1d-4297-ae38-0cee6ca3696a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.413270 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.413271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g9lv\" (UniqueName: \"kubernetes.io/projected/523a084f-cd1d-4297-ae38-0cee6ca3696a-kube-api-access-7g9lv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.413492 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.413307    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.413492 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.413358    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.413492 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.413395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.413492 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.413481    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.514607 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.514507    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.514607 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.514561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.514607 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.514597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.514930 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.514711    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.514930 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.514782    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/523a084f-cd1d-4297-ae38-0cee6ca3696a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.514930 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.514805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g9lv\" (UniqueName: \"kubernetes.io/projected/523a084f-cd1d-4297-ae38-0cee6ca3696a-kube-api-access-7g9lv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.515074 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.514996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.515074 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.515058    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.515165 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.515145    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.516967 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.516934    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/523a084f-cd1d-4297-ae38-0cee6ca3696a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.517179 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.517161    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/523a084f-cd1d-4297-ae38-0cee6ca3696a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.524549 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.524520    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g9lv\" (UniqueName: \"kubernetes.io/projected/523a084f-cd1d-4297-ae38-0cee6ca3696a-kube-api-access-7g9lv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n\" (UID: \"523a084f-cd1d-4297-ae38-0cee6ca3696a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.556957 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.556918    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:03.843558 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:03.843517    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"]
Sep 11 13:01:03.848774 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:01:03.848746    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod523a084f_cd1d_4297_ae38_0cee6ca3696a.slice/crio-61121d76c199216bd13dd931a7394f5c05e570b5408ed3fb6006996c62ba667c WatchSource:0}: Error finding container 61121d76c199216bd13dd931a7394f5c05e570b5408ed3fb6006996c62ba667c: Status 404 returned error can't find the container with id 61121d76c199216bd13dd931a7394f5c05e570b5408ed3fb6006996c62ba667c
Sep 11 13:01:04.182809 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:04.182773    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n" event={"ID":"523a084f-cd1d-4297-ae38-0cee6ca3696a","Type":"ContainerStarted","Data":"5aac7e3496b915b442a4cf009b2d7660005861c1b8a158c87ee3c5fc86f7bd36"}
Sep 11 13:01:04.182954 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:04.182827    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n" event={"ID":"523a084f-cd1d-4297-ae38-0cee6ca3696a","Type":"ContainerStarted","Data":"61121d76c199216bd13dd931a7394f5c05e570b5408ed3fb6006996c62ba667c"}
Sep 11 13:01:04.205742 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:04.205603    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n" podStartSLOduration=1.014721291 podStartE2EDuration="1.20558763s" totalImagesPullingTime="190.866339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:01:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:01:04.205385657 +0000 UTC m=+832.267692626" watchObservedRunningTime="2026-09-11 13:01:04.20558763 +0000 UTC m=+832.267894588"
Sep 11 13:01:06.928442 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:06.928407    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:01:06.928977 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:06.928478    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:01:06.941389 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:06.941352    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:01:07.210315 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:07.210219    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pc5w5"
Sep 11 13:01:13.557860 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:13.557816    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:13.558293 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:13.557899    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:13.572953 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:13.572918    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:01:14.247891 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:01:14.247861    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n"
Sep 11 13:02:12.465357 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:12.465271    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:02:12.469382 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:12.469357    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:02:27.807714 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:27.807672    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66d75f6c7-svpz4"]
Sep 11 13:02:27.808140 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:27.807931    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66d75f6c7-svpz4" podUID="b4034f90-8419-46e0-b892-da93843a3bc0" containerName="authorino" containerID="cri-o://aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5" gracePeriod=30
Sep 11 13:02:28.059854 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.059786    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d75f6c7-svpz4"
Sep 11 13:02:28.082964 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.082926    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b4034f90-8419-46e0-b892-da93843a3bc0-tls-cert\") pod \"b4034f90-8419-46e0-b892-da93843a3bc0\" (UID: \"b4034f90-8419-46e0-b892-da93843a3bc0\") "
Sep 11 13:02:28.083151 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.083040    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjjj7\" (UniqueName: \"kubernetes.io/projected/b4034f90-8419-46e0-b892-da93843a3bc0-kube-api-access-rjjj7\") pod \"b4034f90-8419-46e0-b892-da93843a3bc0\" (UID: \"b4034f90-8419-46e0-b892-da93843a3bc0\") "
Sep 11 13:02:28.093862 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.093818    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4034f90-8419-46e0-b892-da93843a3bc0-kube-api-access-rjjj7" (OuterVolumeSpecName: "kube-api-access-rjjj7") pod "b4034f90-8419-46e0-b892-da93843a3bc0" (UID: "b4034f90-8419-46e0-b892-da93843a3bc0"). InnerVolumeSpecName "kube-api-access-rjjj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:02:28.100999 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.100961    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4034f90-8419-46e0-b892-da93843a3bc0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b4034f90-8419-46e0-b892-da93843a3bc0" (UID: "b4034f90-8419-46e0-b892-da93843a3bc0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:02:28.185321 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.185272    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b4034f90-8419-46e0-b892-da93843a3bc0-tls-cert\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:02:28.185321 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.185312    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjjj7\" (UniqueName: \"kubernetes.io/projected/b4034f90-8419-46e0-b892-da93843a3bc0-kube-api-access-rjjj7\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:02:28.617612 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.617579    2564 generic.go:356] "Generic (PLEG): container finished" podID="b4034f90-8419-46e0-b892-da93843a3bc0" containerID="aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5" exitCode=0
Sep 11 13:02:28.617824 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.617634    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d75f6c7-svpz4" event={"ID":"b4034f90-8419-46e0-b892-da93843a3bc0","Type":"ContainerDied","Data":"aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5"}
Sep 11 13:02:28.617824 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.617659    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d75f6c7-svpz4"
Sep 11 13:02:28.617824 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.617685    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d75f6c7-svpz4" event={"ID":"b4034f90-8419-46e0-b892-da93843a3bc0","Type":"ContainerDied","Data":"68788bd75f9014853b812eaaebb50f99f230ac6c5815dc3411779a4bcfd4b46f"}
Sep 11 13:02:28.617824 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.617702    2564 scope.go:117] "RemoveContainer" containerID="aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5"
Sep 11 13:02:28.626757 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.626736    2564 scope.go:117] "RemoveContainer" containerID="aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5"
Sep 11 13:02:28.627107 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:02:28.627084    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5\": container with ID starting with aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5 not found: ID does not exist" containerID="aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5"
Sep 11 13:02:28.627210 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.627116    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5"} err="failed to get container status \"aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5\": rpc error: code = NotFound desc = could not find container \"aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5\": container with ID starting with aacece2d1d29a82830812e8ffee293a730de6934b6f3d00fcea8a2bdd49b6ae5 not found: ID does not exist"
Sep 11 13:02:28.652921 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.652883    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66d75f6c7-svpz4"]
Sep 11 13:02:28.662154 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:28.662120    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66d75f6c7-svpz4"]
Sep 11 13:02:30.484165 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:30.484129    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4034f90-8419-46e0-b892-da93843a3bc0" path="/var/lib/kubelet/pods/b4034f90-8419-46e0-b892-da93843a3bc0/volumes"
Sep 11 13:02:50.049961 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.049922    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"]
Sep 11 13:02:50.050576 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.050385    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4034f90-8419-46e0-b892-da93843a3bc0" containerName="authorino"
Sep 11 13:02:50.050576 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.050399    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4034f90-8419-46e0-b892-da93843a3bc0" containerName="authorino"
Sep 11 13:02:50.050576 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.050472    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4034f90-8419-46e0-b892-da93843a3bc0" containerName="authorino"
Sep 11 13:02:50.053704 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.053681    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.056950 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.056920    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-0e2299-serving-cert\""
Sep 11 13:02:50.069404 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.069374    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"]
Sep 11 13:02:50.195039 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.195001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c84452e-2be8-4b4e-828c-b06abad45818-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.195237 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.195058    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-metrics-certs\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.195237 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.195119    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkfd2\" (UniqueName: \"kubernetes.io/projected/0c84452e-2be8-4b4e-828c-b06abad45818-kube-api-access-gkfd2\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.195237 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.195170    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-maas-api-tls\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.296627 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.296586    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c84452e-2be8-4b4e-828c-b06abad45818-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.296627 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.296628    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-metrics-certs\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.296851 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.296668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkfd2\" (UniqueName: \"kubernetes.io/projected/0c84452e-2be8-4b4e-828c-b06abad45818-kube-api-access-gkfd2\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.296851 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.296709    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-maas-api-tls\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.297432 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.297405    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c84452e-2be8-4b4e-828c-b06abad45818-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.299206 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.299182    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-metrics-certs\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.299303 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.299271    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-maas-api-tls\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.305216 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.305144    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkfd2\" (UniqueName: \"kubernetes.io/projected/0c84452e-2be8-4b4e-828c-b06abad45818-kube-api-access-gkfd2\") pod \"maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.369123 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.369089    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:50.648675 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.648632    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"]
Sep 11 13:02:50.654775 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:02:50.654682    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c84452e_2be8_4b4e_828c_b06abad45818.slice/crio-438e137a8d05f6d9f28d1a8cffcd128ec2c4000c961d9e498ebbfc0419ca4f75 WatchSource:0}: Error finding container 438e137a8d05f6d9f28d1a8cffcd128ec2c4000c961d9e498ebbfc0419ca4f75: Status 404 returned error can't find the container with id 438e137a8d05f6d9f28d1a8cffcd128ec2c4000c961d9e498ebbfc0419ca4f75
Sep 11 13:02:50.738560 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.738519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p" event={"ID":"0c84452e-2be8-4b4e-828c-b06abad45818","Type":"ContainerStarted","Data":"438e137a8d05f6d9f28d1a8cffcd128ec2c4000c961d9e498ebbfc0419ca4f75"}
Sep 11 13:02:50.865596 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.865554    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"]
Sep 11 13:02:50.870204 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.870178    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:02:50.874614 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.874580    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-0e2299\""
Sep 11 13:02:50.874785 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.874580    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-0e2299-dockercfg-jjvhn\""
Sep 11 13:02:50.882401 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.882372    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"]
Sep 11 13:02:50.917827 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.917746    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"]
Sep 11 13:02:50.921164 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.921137    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:02:50.930254 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:50.930224    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"]
Sep 11 13:02:51.003853 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.003811    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-555xb\" (UniqueName: \"kubernetes.io/projected/3494cc44-80ed-4561-b6be-02462497f361-kube-api-access-555xb\") pod \"payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr\" (UID: \"3494cc44-80ed-4561-b6be-02462497f361\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:02:51.004047 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.003999    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3494cc44-80ed-4561-b6be-02462497f361-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr\" (UID: \"3494cc44-80ed-4561-b6be-02462497f361\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:02:51.105352 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.105317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9m56\" (UniqueName: \"kubernetes.io/projected/7c06f7d7-14ea-424b-a67e-2fbe560b5003-kube-api-access-n9m56\") pod \"payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh\" (UID: \"7c06f7d7-14ea-424b-a67e-2fbe560b5003\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:02:51.105807 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.105358    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3494cc44-80ed-4561-b6be-02462497f361-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr\" (UID: \"3494cc44-80ed-4561-b6be-02462497f361\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:02:51.105807 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.105459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-555xb\" (UniqueName: \"kubernetes.io/projected/3494cc44-80ed-4561-b6be-02462497f361-kube-api-access-555xb\") pod \"payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr\" (UID: \"3494cc44-80ed-4561-b6be-02462497f361\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:02:51.105807 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.105522    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c06f7d7-14ea-424b-a67e-2fbe560b5003-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh\" (UID: \"7c06f7d7-14ea-424b-a67e-2fbe560b5003\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:02:51.106172 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.106153    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3494cc44-80ed-4561-b6be-02462497f361-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr\" (UID: \"3494cc44-80ed-4561-b6be-02462497f361\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:02:51.113623 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.113589    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-555xb\" (UniqueName: \"kubernetes.io/projected/3494cc44-80ed-4561-b6be-02462497f361-kube-api-access-555xb\") pod \"payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr\" (UID: \"3494cc44-80ed-4561-b6be-02462497f361\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:02:51.183353 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.183200    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:02:51.206797 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.206753    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c06f7d7-14ea-424b-a67e-2fbe560b5003-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh\" (UID: \"7c06f7d7-14ea-424b-a67e-2fbe560b5003\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:02:51.206944 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.206863    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9m56\" (UniqueName: \"kubernetes.io/projected/7c06f7d7-14ea-424b-a67e-2fbe560b5003-kube-api-access-n9m56\") pod \"payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh\" (UID: \"7c06f7d7-14ea-424b-a67e-2fbe560b5003\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:02:51.207691 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.207661    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c06f7d7-14ea-424b-a67e-2fbe560b5003-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh\" (UID: \"7c06f7d7-14ea-424b-a67e-2fbe560b5003\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:02:51.220676 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.220595    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9m56\" (UniqueName: \"kubernetes.io/projected/7c06f7d7-14ea-424b-a67e-2fbe560b5003-kube-api-access-n9m56\") pod \"payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh\" (UID: \"7c06f7d7-14ea-424b-a67e-2fbe560b5003\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:02:51.233369 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.233335    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:02:51.471731 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.471653    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"]
Sep 11 13:02:51.474124 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:02:51.474092    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3494cc44_80ed_4561_b6be_02462497f361.slice/crio-97a227faeab5ea3a1674c10d707d189f27d54418cd9aa3f0532998c27c5ea07e WatchSource:0}: Error finding container 97a227faeab5ea3a1674c10d707d189f27d54418cd9aa3f0532998c27c5ea07e: Status 404 returned error can't find the container with id 97a227faeab5ea3a1674c10d707d189f27d54418cd9aa3f0532998c27c5ea07e
Sep 11 13:02:51.536598 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.536571    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"]
Sep 11 13:02:51.539616 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:02:51.539571    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c06f7d7_14ea_424b_a67e_2fbe560b5003.slice/crio-fd5093d0eec02baf8c6bfc776b9064212c72a52817a7cd9735debd9008805726 WatchSource:0}: Error finding container fd5093d0eec02baf8c6bfc776b9064212c72a52817a7cd9735debd9008805726: Status 404 returned error can't find the container with id fd5093d0eec02baf8c6bfc776b9064212c72a52817a7cd9735debd9008805726
Sep 11 13:02:51.749211 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.749106    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p" event={"ID":"0c84452e-2be8-4b4e-828c-b06abad45818","Type":"ContainerStarted","Data":"02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da"}
Sep 11 13:02:51.749536 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.749247    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:51.752071 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.751925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh" event={"ID":"7c06f7d7-14ea-424b-a67e-2fbe560b5003","Type":"ContainerStarted","Data":"0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e"}
Sep 11 13:02:51.752071 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.752033    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh" event={"ID":"7c06f7d7-14ea-424b-a67e-2fbe560b5003","Type":"ContainerStarted","Data":"fd5093d0eec02baf8c6bfc776b9064212c72a52817a7cd9735debd9008805726"}
Sep 11 13:02:51.752761 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.752669    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:02:51.754330 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.754255    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr" event={"ID":"3494cc44-80ed-4561-b6be-02462497f361","Type":"ContainerStarted","Data":"159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04"}
Sep 11 13:02:51.754330 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.754305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr" event={"ID":"3494cc44-80ed-4561-b6be-02462497f361","Type":"ContainerStarted","Data":"97a227faeab5ea3a1674c10d707d189f27d54418cd9aa3f0532998c27c5ea07e"}
Sep 11 13:02:51.754561 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.754334    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:02:51.772811 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.772752    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p" podStartSLOduration=1.284328683 podStartE2EDuration="1.772731713s" totalImagesPullingTime="488.40303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:02:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:02:51.769889883 +0000 UTC m=+939.832196884" watchObservedRunningTime="2026-09-11 13:02:51.772731713 +0000 UTC m=+939.835038741"
Sep 11 13:02:51.792711 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.792657    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr" podStartSLOduration=1.792624613 podStartE2EDuration="1.792624613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:02:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:02:51.791957307 +0000 UTC m=+939.854264262" watchObservedRunningTime="2026-09-11 13:02:51.792624613 +0000 UTC m=+939.854931571"
Sep 11 13:02:51.811900 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:51.811846    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh" podStartSLOduration=1.811829539 podStartE2EDuration="1.811829539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:02:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:02:51.810752582 +0000 UTC m=+939.873059541" watchObservedRunningTime="2026-09-11 13:02:51.811829539 +0000 UTC m=+939.874136496"
Sep 11 13:02:57.513798 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.513754    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"]
Sep 11 13:02:57.518308 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.518289    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.521768 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.521720    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d473235f-serving-cert\""
Sep 11 13:02:57.536337 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.535485    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"]
Sep 11 13:02:57.564139 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.564092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjtx4\" (UniqueName: \"kubernetes.io/projected/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-kube-api-access-xjtx4\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.564340 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.564175    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-openshift-service-ca\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.564340 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.564254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-metrics-certs\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.564340 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.564309    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-maas-api-tls\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.664848 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.664816    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-maas-api-tls\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.665048 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.664880    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjtx4\" (UniqueName: \"kubernetes.io/projected/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-kube-api-access-xjtx4\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.665048 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.664909    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-openshift-service-ca\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.665048 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.664946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-metrics-certs\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.665048 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:02:57.664979    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-serving-cert: secret "maas-api-e2e-ait-d473235f-serving-cert" not found
Sep 11 13:02:57.665048 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:02:57.665044    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-maas-api-tls podName:84d1b945-8c2a-42fe-b5b9-c85678f5f6ae nodeName:}" failed. No retries permitted until 2026-09-11 13:02:58.165028091 +0000 UTC m=+946.227335026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-maas-api-tls") pod "maas-api-e2e-ait-d473235f-69c6767d8b-czrf8" (UID: "84d1b945-8c2a-42fe-b5b9-c85678f5f6ae") : secret "maas-api-e2e-ait-d473235f-serving-cert" not found
Sep 11 13:02:57.665764 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.665730    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-openshift-service-ca\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.667534 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.667505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-metrics-certs\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.675574 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.675548    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjtx4\" (UniqueName: \"kubernetes.io/projected/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-kube-api-access-xjtx4\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:57.765702 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:57.765604    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:02:58.168111 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.168080    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-maas-api-tls\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:58.170670 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.170610    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-maas-api-tls\") pod \"maas-api-e2e-ait-d473235f-69c6767d8b-czrf8\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:58.243163 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.243115    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"]
Sep 11 13:02:58.247222 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.247196    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:02:58.250522 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.250496    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d473235f\""
Sep 11 13:02:58.250682 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.250552    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d473235f-dockercfg-wqwst\""
Sep 11 13:02:58.262542 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.262508    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"]
Sep 11 13:02:58.269388 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.269357    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e3607da-1712-4af3-8e30-648fecb7a210-plugins-config-volume\") pod \"payload-processing-e2e-ait-d473235f-f67947987-xkcfk\" (UID: \"0e3607da-1712-4af3-8e30-648fecb7a210\") " pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:02:58.269523 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.269398    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q4n6\" (UniqueName: \"kubernetes.io/projected/0e3607da-1712-4af3-8e30-648fecb7a210-kube-api-access-5q4n6\") pod \"payload-processing-e2e-ait-d473235f-f67947987-xkcfk\" (UID: \"0e3607da-1712-4af3-8e30-648fecb7a210\") " pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:02:58.292684 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.292635    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"]
Sep 11 13:02:58.296871 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.296849    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:02:58.308353 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.308325    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"]
Sep 11 13:02:58.370728 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.370696    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxnnd\" (UniqueName: \"kubernetes.io/projected/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-kube-api-access-kxnnd\") pod \"payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx\" (UID: \"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:02:58.370907 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.370762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx\" (UID: \"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:02:58.370907 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.370852    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e3607da-1712-4af3-8e30-648fecb7a210-plugins-config-volume\") pod \"payload-processing-e2e-ait-d473235f-f67947987-xkcfk\" (UID: \"0e3607da-1712-4af3-8e30-648fecb7a210\") " pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:02:58.370907 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.370893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q4n6\" (UniqueName: \"kubernetes.io/projected/0e3607da-1712-4af3-8e30-648fecb7a210-kube-api-access-5q4n6\") pod \"payload-processing-e2e-ait-d473235f-f67947987-xkcfk\" (UID: \"0e3607da-1712-4af3-8e30-648fecb7a210\") " pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:02:58.371531 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.371511    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e3607da-1712-4af3-8e30-648fecb7a210-plugins-config-volume\") pod \"payload-processing-e2e-ait-d473235f-f67947987-xkcfk\" (UID: \"0e3607da-1712-4af3-8e30-648fecb7a210\") " pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:02:58.381200 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.381143    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q4n6\" (UniqueName: \"kubernetes.io/projected/0e3607da-1712-4af3-8e30-648fecb7a210-kube-api-access-5q4n6\") pod \"payload-processing-e2e-ait-d473235f-f67947987-xkcfk\" (UID: \"0e3607da-1712-4af3-8e30-648fecb7a210\") " pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:02:58.431979 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.431882    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:58.471758 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.471719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx\" (UID: \"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:02:58.471961 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.471824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxnnd\" (UniqueName: \"kubernetes.io/projected/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-kube-api-access-kxnnd\") pod \"payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx\" (UID: \"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:02:58.472457 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.472432    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx\" (UID: \"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:02:58.484798 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.484773    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxnnd\" (UniqueName: \"kubernetes.io/projected/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-kube-api-access-kxnnd\") pod \"payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx\" (UID: \"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:02:58.558406 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.558363    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:02:58.608076 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.608040    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:02:58.712209 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.712178    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"]
Sep 11 13:02:58.714385 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:02:58.714354    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d1b945_8c2a_42fe_b5b9_c85678f5f6ae.slice/crio-1296c56f16d10e3d4a3cf68a9f0237a531880f9b611ebde92c374186586f54e4 WatchSource:0}: Error finding container 1296c56f16d10e3d4a3cf68a9f0237a531880f9b611ebde92c374186586f54e4: Status 404 returned error can't find the container with id 1296c56f16d10e3d4a3cf68a9f0237a531880f9b611ebde92c374186586f54e4
Sep 11 13:02:58.797452 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.797417    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8" event={"ID":"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae","Type":"ContainerStarted","Data":"1296c56f16d10e3d4a3cf68a9f0237a531880f9b611ebde92c374186586f54e4"}
Sep 11 13:02:58.840825 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.838373    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"]
Sep 11 13:02:58.843729 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:02:58.843685    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3607da_1712_4af3_8e30_648fecb7a210.slice/crio-63ee36ac6e2ac698a22c28f93bee5e9ccb208f936e08aa28249a7abfb0b9f61b WatchSource:0}: Error finding container 63ee36ac6e2ac698a22c28f93bee5e9ccb208f936e08aa28249a7abfb0b9f61b: Status 404 returned error can't find the container with id 63ee36ac6e2ac698a22c28f93bee5e9ccb208f936e08aa28249a7abfb0b9f61b
Sep 11 13:02:58.893692 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:58.893669    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"]
Sep 11 13:02:58.904079 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:02:58.904046    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c37ffae_9bd4_40a5_ad33_0eb82bb744e4.slice/crio-0da0aadbed98868e1a94c177aec445a69ebdac420b839b953dd05b27aef074a4 WatchSource:0}: Error finding container 0da0aadbed98868e1a94c177aec445a69ebdac420b839b953dd05b27aef074a4: Status 404 returned error can't find the container with id 0da0aadbed98868e1a94c177aec445a69ebdac420b839b953dd05b27aef074a4
Sep 11 13:02:59.804178 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.804099    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8" event={"ID":"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae","Type":"ContainerStarted","Data":"33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4"}
Sep 11 13:02:59.804663 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.804258    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:02:59.805753 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.805715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx" event={"ID":"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4","Type":"ContainerStarted","Data":"1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d"}
Sep 11 13:02:59.805864 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.805765    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx" event={"ID":"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4","Type":"ContainerStarted","Data":"0da0aadbed98868e1a94c177aec445a69ebdac420b839b953dd05b27aef074a4"}
Sep 11 13:02:59.805864 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.805787    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:02:59.807325 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.807301    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk" event={"ID":"0e3607da-1712-4af3-8e30-648fecb7a210","Type":"ContainerStarted","Data":"66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb"}
Sep 11 13:02:59.807438 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.807339    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk" event={"ID":"0e3607da-1712-4af3-8e30-648fecb7a210","Type":"ContainerStarted","Data":"63ee36ac6e2ac698a22c28f93bee5e9ccb208f936e08aa28249a7abfb0b9f61b"}
Sep 11 13:02:59.807438 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.807358    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:02:59.832515 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.832460    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8" podStartSLOduration=2.5300204109999997 podStartE2EDuration="2.832442849s" totalImagesPullingTime="302.422438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:02:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:02:59.829875646 +0000 UTC m=+947.892182617" watchObservedRunningTime="2026-09-11 13:02:59.832442849 +0000 UTC m=+947.894749807"
Sep 11 13:02:59.861665 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.861582    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk" podStartSLOduration=1.861560008 podStartE2EDuration="1.861560008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:02:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:02:59.861245767 +0000 UTC m=+947.923552892" watchObservedRunningTime="2026-09-11 13:02:59.861560008 +0000 UTC m=+947.923866967"
Sep 11 13:02:59.887109 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:02:59.887043    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx" podStartSLOduration=1.8870219499999998 podStartE2EDuration="1.88702195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:02:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:02:59.884405255 +0000 UTC m=+947.946712213" watchObservedRunningTime="2026-09-11 13:02:59.88702195 +0000 UTC m=+947.949328911"
Sep 11 13:03:01.065828 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.065794    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"]
Sep 11 13:03:01.069724 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.069690    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:03:01.073299 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.073267    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-5f1b61\""
Sep 11 13:03:01.081389 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.081362    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-5f1b61-dockercfg-4xkb5\""
Sep 11 13:03:01.096312 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.096276    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"]
Sep 11 13:03:01.099989 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.099956    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt4lm\" (UniqueName: \"kubernetes.io/projected/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-kube-api-access-jt4lm\") pod \"payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk\" (UID: \"21d92bf5-4c7e-439f-ab2c-caa064ce9e82\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:03:01.100158 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.099997    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk\" (UID: \"21d92bf5-4c7e-439f-ab2c-caa064ce9e82\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:03:01.200895 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.200858    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt4lm\" (UniqueName: \"kubernetes.io/projected/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-kube-api-access-jt4lm\") pod \"payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk\" (UID: \"21d92bf5-4c7e-439f-ab2c-caa064ce9e82\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:03:01.201110 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.200906    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk\" (UID: \"21d92bf5-4c7e-439f-ab2c-caa064ce9e82\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:03:01.201717 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.201691    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk\" (UID: \"21d92bf5-4c7e-439f-ab2c-caa064ce9e82\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:03:01.212367 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.212334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt4lm\" (UniqueName: \"kubernetes.io/projected/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-kube-api-access-jt4lm\") pod \"payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk\" (UID: \"21d92bf5-4c7e-439f-ab2c-caa064ce9e82\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:03:01.384477 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.384437    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:03:01.664583 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.664433    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"]
Sep 11 13:03:01.667043 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:03:01.667010    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d92bf5_4c7e_439f_ab2c_caa064ce9e82.slice/crio-f1c4816ac46e41e4d08db59d5e8780d4ceae841baedf0a42c6423cda6974ce1d WatchSource:0}: Error finding container f1c4816ac46e41e4d08db59d5e8780d4ceae841baedf0a42c6423cda6974ce1d: Status 404 returned error can't find the container with id f1c4816ac46e41e4d08db59d5e8780d4ceae841baedf0a42c6423cda6974ce1d
Sep 11 13:03:01.821342 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.821303    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk" event={"ID":"21d92bf5-4c7e-439f-ab2c-caa064ce9e82","Type":"ContainerStarted","Data":"a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b"}
Sep 11 13:03:01.821541 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.821364    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk" event={"ID":"21d92bf5-4c7e-439f-ab2c-caa064ce9e82","Type":"ContainerStarted","Data":"f1c4816ac46e41e4d08db59d5e8780d4ceae841baedf0a42c6423cda6974ce1d"}
Sep 11 13:03:01.821541 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.821389    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:03:01.851426 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:01.851362    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk" podStartSLOduration=0.851342864 podStartE2EDuration="851.342864ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:03:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:03:01.849926641 +0000 UTC m=+949.912233599" watchObservedRunningTime="2026-09-11 13:03:01.851342864 +0000 UTC m=+949.913649822"
Sep 11 13:03:02.762926 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:02.762897    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:03:03.768321 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:03.768291    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:03:05.819345 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:05.819316    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:03:10.816184 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:10.816147    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:03:10.816676 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:10.816390    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:03:12.831498 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:12.831470    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:03:19.524133 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.524099    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"]
Sep 11 13:03:19.529222 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.529197    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.532922 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.532899    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-de3842-serving-cert\""
Sep 11 13:03:19.542160 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.542132    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"]
Sep 11 13:03:19.687618 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.687571    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgfjp\" (UniqueName: \"kubernetes.io/projected/9ddbfdb8-89fe-4950-83ae-639fde042751-kube-api-access-lgfjp\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.687618 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.687626    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-maas-api-tls\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.688000 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.687756    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ddbfdb8-89fe-4950-83ae-639fde042751-openshift-service-ca\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.688000 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.687786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-metrics-certs\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.788328 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.788235    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgfjp\" (UniqueName: \"kubernetes.io/projected/9ddbfdb8-89fe-4950-83ae-639fde042751-kube-api-access-lgfjp\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.788328 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.788277    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-maas-api-tls\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.788328 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.788312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ddbfdb8-89fe-4950-83ae-639fde042751-openshift-service-ca\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.788328 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.788328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-metrics-certs\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.789053 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.789024    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ddbfdb8-89fe-4950-83ae-639fde042751-openshift-service-ca\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.790904 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.790876    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-metrics-certs\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.790998 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.790916    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-maas-api-tls\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.796960 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.796932    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgfjp\" (UniqueName: \"kubernetes.io/projected/9ddbfdb8-89fe-4950-83ae-639fde042751-kube-api-access-lgfjp\") pod \"maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:19.840010 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:19.839970    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:20.147261 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:03:20.147223    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ddbfdb8_89fe_4950_83ae_639fde042751.slice/crio-dc73b0fb357c831fc4e89d6f5fd7ee7b3264c0a35d79d02b668a12aef9b79606 WatchSource:0}: Error finding container dc73b0fb357c831fc4e89d6f5fd7ee7b3264c0a35d79d02b668a12aef9b79606: Status 404 returned error can't find the container with id dc73b0fb357c831fc4e89d6f5fd7ee7b3264c0a35d79d02b668a12aef9b79606
Sep 11 13:03:20.147261 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.147235    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"]
Sep 11 13:03:20.442227 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.442135    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"]
Sep 11 13:03:20.447701 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.447676    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:03:20.450514 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.450480    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-de3842\""
Sep 11 13:03:20.450668 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.450521    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-de3842-dockercfg-7tktt\""
Sep 11 13:03:20.457339 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.457308    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"]
Sep 11 13:03:20.599249 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.599208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbdb2ef1-7705-430d-8e86-32ccb52a4def-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t\" (UID: \"dbdb2ef1-7705-430d-8e86-32ccb52a4def\") " pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:03:20.599249 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.599251    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkzfb\" (UniqueName: \"kubernetes.io/projected/dbdb2ef1-7705-430d-8e86-32ccb52a4def-kube-api-access-wkzfb\") pod \"payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t\" (UID: \"dbdb2ef1-7705-430d-8e86-32ccb52a4def\") " pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:03:20.701048 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.700952    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbdb2ef1-7705-430d-8e86-32ccb52a4def-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t\" (UID: \"dbdb2ef1-7705-430d-8e86-32ccb52a4def\") " pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:03:20.701048 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.701004    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkzfb\" (UniqueName: \"kubernetes.io/projected/dbdb2ef1-7705-430d-8e86-32ccb52a4def-kube-api-access-wkzfb\") pod \"payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t\" (UID: \"dbdb2ef1-7705-430d-8e86-32ccb52a4def\") " pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:03:20.701805 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.701778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbdb2ef1-7705-430d-8e86-32ccb52a4def-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t\" (UID: \"dbdb2ef1-7705-430d-8e86-32ccb52a4def\") " pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:03:20.709236 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.709208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkzfb\" (UniqueName: \"kubernetes.io/projected/dbdb2ef1-7705-430d-8e86-32ccb52a4def-kube-api-access-wkzfb\") pod \"payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t\" (UID: \"dbdb2ef1-7705-430d-8e86-32ccb52a4def\") " pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:03:20.760169 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.760141    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:03:20.943800 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.943749    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8" event={"ID":"9ddbfdb8-89fe-4950-83ae-639fde042751","Type":"ContainerStarted","Data":"f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec"}
Sep 11 13:03:20.943999 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.943852    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8" event={"ID":"9ddbfdb8-89fe-4950-83ae-639fde042751","Type":"ContainerStarted","Data":"dc73b0fb357c831fc4e89d6f5fd7ee7b3264c0a35d79d02b668a12aef9b79606"}
Sep 11 13:03:20.943999 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.943976    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:20.966650 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:20.966536    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8" podStartSLOduration=1.373356925 podStartE2EDuration="1.966517564s" totalImagesPullingTime="593.160639ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:03:20.964043237 +0000 UTC m=+969.026350195" watchObservedRunningTime="2026-09-11 13:03:20.966517564 +0000 UTC m=+969.028824521"
Sep 11 13:03:21.044681 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:21.044615    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"]
Sep 11 13:03:21.046385 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:03:21.046349    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbdb2ef1_7705_430d_8e86_32ccb52a4def.slice/crio-1d367b17d0bda58e5b917e6814b2c210fd7b4f8ae0acc1532daa49aa3ab1b738 WatchSource:0}: Error finding container 1d367b17d0bda58e5b917e6814b2c210fd7b4f8ae0acc1532daa49aa3ab1b738: Status 404 returned error can't find the container with id 1d367b17d0bda58e5b917e6814b2c210fd7b4f8ae0acc1532daa49aa3ab1b738
Sep 11 13:03:21.952389 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:21.952351    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t" event={"ID":"dbdb2ef1-7705-430d-8e86-32ccb52a4def","Type":"ContainerStarted","Data":"6c68e3abf5a4d754be9f53de71b2b33e58f499e0d6cb369248ca95276997b5dd"}
Sep 11 13:03:21.952867 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:21.952547    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:03:21.952867 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:21.952580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t" event={"ID":"dbdb2ef1-7705-430d-8e86-32ccb52a4def","Type":"ContainerStarted","Data":"1d367b17d0bda58e5b917e6814b2c210fd7b4f8ae0acc1532daa49aa3ab1b738"}
Sep 11 13:03:21.971308 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:21.971253    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t" podStartSLOduration=1.971234597 podStartE2EDuration="1.971234597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:03:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:03:21.968866116 +0000 UTC m=+970.031173088" watchObservedRunningTime="2026-09-11 13:03:21.971234597 +0000 UTC m=+970.033541574"
Sep 11 13:03:26.957938 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:26.957905    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:03:32.961225 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:32.961192    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:03:38.450841 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.450750    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"]
Sep 11 13:03:38.451306 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.451221    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8" podUID="84d1b945-8c2a-42fe-b5b9-c85678f5f6ae" containerName="maas-api" containerID="cri-o://33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4" gracePeriod=30
Sep 11 13:03:38.520928 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.520887    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"]
Sep 11 13:03:38.521437 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.521382    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk" podUID="0e3607da-1712-4af3-8e30-648fecb7a210" containerName="payload-processing" containerID="cri-o://66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb" gracePeriod=30
Sep 11 13:03:38.534903 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.534867    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"]
Sep 11 13:03:38.535176 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.535147    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx" podUID="5c37ffae-9bd4-40a5-ad33-0eb82bb744e4" containerName="payload-pre-processing" containerID="cri-o://1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d" gracePeriod=30
Sep 11 13:03:38.834321 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.834026    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:03:38.868910 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.868883    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:03:38.877975 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.877934    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:03:38.881160 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.881129    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e3607da-1712-4af3-8e30-648fecb7a210-plugins-config-volume\") pod \"0e3607da-1712-4af3-8e30-648fecb7a210\" (UID: \"0e3607da-1712-4af3-8e30-648fecb7a210\") "
Sep 11 13:03:38.881393 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.881377    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q4n6\" (UniqueName: \"kubernetes.io/projected/0e3607da-1712-4af3-8e30-648fecb7a210-kube-api-access-5q4n6\") pod \"0e3607da-1712-4af3-8e30-648fecb7a210\" (UID: \"0e3607da-1712-4af3-8e30-648fecb7a210\") "
Sep 11 13:03:38.881526 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.881513    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-openshift-service-ca\") pod \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") "
Sep 11 13:03:38.881772 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.881756    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-metrics-certs\") pod \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") "
Sep 11 13:03:38.881908 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.881896    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-maas-api-tls\") pod \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") "
Sep 11 13:03:38.882059 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.882043    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjtx4\" (UniqueName: \"kubernetes.io/projected/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-kube-api-access-xjtx4\") pod \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\" (UID: \"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae\") "
Sep 11 13:03:38.882958 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.882920    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3607da-1712-4af3-8e30-648fecb7a210-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0e3607da-1712-4af3-8e30-648fecb7a210" (UID: "0e3607da-1712-4af3-8e30-648fecb7a210"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:03:38.883804 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.883775    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "84d1b945-8c2a-42fe-b5b9-c85678f5f6ae" (UID: "84d1b945-8c2a-42fe-b5b9-c85678f5f6ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:03:38.885149 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.885117    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e3607da-1712-4af3-8e30-648fecb7a210-kube-api-access-5q4n6" (OuterVolumeSpecName: "kube-api-access-5q4n6") pod "0e3607da-1712-4af3-8e30-648fecb7a210" (UID: "0e3607da-1712-4af3-8e30-648fecb7a210"). InnerVolumeSpecName "kube-api-access-5q4n6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:03:38.885871 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.885834    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "84d1b945-8c2a-42fe-b5b9-c85678f5f6ae" (UID: "84d1b945-8c2a-42fe-b5b9-c85678f5f6ae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:03:38.886459 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.886409    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-kube-api-access-xjtx4" (OuterVolumeSpecName: "kube-api-access-xjtx4") pod "84d1b945-8c2a-42fe-b5b9-c85678f5f6ae" (UID: "84d1b945-8c2a-42fe-b5b9-c85678f5f6ae"). InnerVolumeSpecName "kube-api-access-xjtx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:03:38.887591 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.887556    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "84d1b945-8c2a-42fe-b5b9-c85678f5f6ae" (UID: "84d1b945-8c2a-42fe-b5b9-c85678f5f6ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:03:38.983949 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.983858    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-plugins-config-volume\") pod \"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4\" (UID: \"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4\") "
Sep 11 13:03:38.984113 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.983975    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxnnd\" (UniqueName: \"kubernetes.io/projected/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-kube-api-access-kxnnd\") pod \"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4\" (UID: \"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4\") "
Sep 11 13:03:38.984197 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.984186    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:03:38.984257 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.984197    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:03:38.984257 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.984206    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjtx4\" (UniqueName: \"kubernetes.io/projected/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-kube-api-access-xjtx4\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:03:38.984257 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.984215    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e3607da-1712-4af3-8e30-648fecb7a210-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:03:38.984257 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.984225    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q4n6\" (UniqueName: \"kubernetes.io/projected/0e3607da-1712-4af3-8e30-648fecb7a210-kube-api-access-5q4n6\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:03:38.984257 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.984234    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:03:38.984457 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.984325    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5c37ffae-9bd4-40a5-ad33-0eb82bb744e4" (UID: "5c37ffae-9bd4-40a5-ad33-0eb82bb744e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:03:38.986249 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:38.986220    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-kube-api-access-kxnnd" (OuterVolumeSpecName: "kube-api-access-kxnnd") pod "5c37ffae-9bd4-40a5-ad33-0eb82bb744e4" (UID: "5c37ffae-9bd4-40a5-ad33-0eb82bb744e4"). InnerVolumeSpecName "kube-api-access-kxnnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:03:39.056831 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.056796    2564 generic.go:356] "Generic (PLEG): container finished" podID="84d1b945-8c2a-42fe-b5b9-c85678f5f6ae" containerID="33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4" exitCode=0
Sep 11 13:03:39.057004 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.056856    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"
Sep 11 13:03:39.057004 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.056884    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8" event={"ID":"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae","Type":"ContainerDied","Data":"33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4"}
Sep 11 13:03:39.057004 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.056972    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8" event={"ID":"84d1b945-8c2a-42fe-b5b9-c85678f5f6ae","Type":"ContainerDied","Data":"1296c56f16d10e3d4a3cf68a9f0237a531880f9b611ebde92c374186586f54e4"}
Sep 11 13:03:39.057142 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.057007    2564 scope.go:117] "RemoveContainer" containerID="33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4"
Sep 11 13:03:39.058486 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.058459    2564 generic.go:356] "Generic (PLEG): container finished" podID="5c37ffae-9bd4-40a5-ad33-0eb82bb744e4" containerID="1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d" exitCode=0
Sep 11 13:03:39.058608 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.058491    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx" event={"ID":"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4","Type":"ContainerDied","Data":"1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d"}
Sep 11 13:03:39.058721 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.058615    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx" event={"ID":"5c37ffae-9bd4-40a5-ad33-0eb82bb744e4","Type":"ContainerDied","Data":"0da0aadbed98868e1a94c177aec445a69ebdac420b839b953dd05b27aef074a4"}
Sep 11 13:03:39.058721 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.058510    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"
Sep 11 13:03:39.059960 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.059939    2564 generic.go:356] "Generic (PLEG): container finished" podID="0e3607da-1712-4af3-8e30-648fecb7a210" containerID="66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb" exitCode=0
Sep 11 13:03:39.060086 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.059995    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"
Sep 11 13:03:39.060086 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.060021    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk" event={"ID":"0e3607da-1712-4af3-8e30-648fecb7a210","Type":"ContainerDied","Data":"66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb"}
Sep 11 13:03:39.060086 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.060063    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk" event={"ID":"0e3607da-1712-4af3-8e30-648fecb7a210","Type":"ContainerDied","Data":"63ee36ac6e2ac698a22c28f93bee5e9ccb208f936e08aa28249a7abfb0b9f61b"}
Sep 11 13:03:39.069466 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.069441    2564 scope.go:117] "RemoveContainer" containerID="33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4"
Sep 11 13:03:39.069932 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:03:39.069909    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4\": container with ID starting with 33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4 not found: ID does not exist" containerID="33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4"
Sep 11 13:03:39.070039 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.069941    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4"} err="failed to get container status \"33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4\": rpc error: code = NotFound desc = could not find container \"33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4\": container with ID starting with 33beb521321d8fbb6df37777ba6550b70726bbcfd1a12db9ad999a2dcdab35b4 not found: ID does not exist"
Sep 11 13:03:39.070039 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.069959    2564 scope.go:117] "RemoveContainer" containerID="1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d"
Sep 11 13:03:39.079546 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.079521    2564 scope.go:117] "RemoveContainer" containerID="1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d"
Sep 11 13:03:39.079922 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:03:39.079902    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d\": container with ID starting with 1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d not found: ID does not exist" containerID="1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d"
Sep 11 13:03:39.079991 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.079931    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d"} err="failed to get container status \"1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d\": rpc error: code = NotFound desc = could not find container \"1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d\": container with ID starting with 1b1627b6356b9b007f5717cd4a86a7d097b8a5aa7a439094312530700ee71b0d not found: ID does not exist"
Sep 11 13:03:39.079991 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.079951    2564 scope.go:117] "RemoveContainer" containerID="66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb"
Sep 11 13:03:39.084494 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.084461    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"]
Sep 11 13:03:39.085053 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.085031    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:03:39.085115 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.085059    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxnnd\" (UniqueName: \"kubernetes.io/projected/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4-kube-api-access-kxnnd\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:03:39.089320 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.089302    2564 scope.go:117] "RemoveContainer" containerID="66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb"
Sep 11 13:03:39.089720 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:03:39.089689    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb\": container with ID starting with 66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb not found: ID does not exist" containerID="66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb"
Sep 11 13:03:39.089805 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.089725    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb"} err="failed to get container status \"66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb\": rpc error: code = NotFound desc = could not find container \"66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb\": container with ID starting with 66d404858d376cea17fbd2f0c764b09d9f6a4e92b4c6465b9ac9992461a72dbb not found: ID does not exist"
Sep 11 13:03:39.089909 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.089890    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d473235f-69c6767d8b-czrf8"]
Sep 11 13:03:39.101401 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.101364    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"]
Sep 11 13:03:39.105148 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.105121    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d473235f-f67947987-xkcfk"]
Sep 11 13:03:39.117305 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.117262    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"]
Sep 11 13:03:39.122238 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:39.122208    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d473235f-756786b9d6-4jsvx"]
Sep 11 13:03:40.483484 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:40.483441    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e3607da-1712-4af3-8e30-648fecb7a210" path="/var/lib/kubelet/pods/0e3607da-1712-4af3-8e30-648fecb7a210/volumes"
Sep 11 13:03:40.484040 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:40.484019    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c37ffae-9bd4-40a5-ad33-0eb82bb744e4" path="/var/lib/kubelet/pods/5c37ffae-9bd4-40a5-ad33-0eb82bb744e4/volumes"
Sep 11 13:03:40.484423 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:03:40.484405    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84d1b945-8c2a-42fe-b5b9-c85678f5f6ae" path="/var/lib/kubelet/pods/84d1b945-8c2a-42fe-b5b9-c85678f5f6ae/volumes"
Sep 11 13:04:42.328678 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:42.328629    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"]
Sep 11 13:04:42.440269 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:42.440225    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk" podUID="21d92bf5-4c7e-439f-ab2c-caa064ce9e82" containerName="payload-pre-processing" containerID="cri-o://a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b" gracePeriod=30
Sep 11 13:04:42.712597 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:42.712566    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:04:42.803468 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:42.803434    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt4lm\" (UniqueName: \"kubernetes.io/projected/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-kube-api-access-jt4lm\") pod \"21d92bf5-4c7e-439f-ab2c-caa064ce9e82\" (UID: \"21d92bf5-4c7e-439f-ab2c-caa064ce9e82\") "
Sep 11 13:04:42.803702 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:42.803517    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-plugins-config-volume\") pod \"21d92bf5-4c7e-439f-ab2c-caa064ce9e82\" (UID: \"21d92bf5-4c7e-439f-ab2c-caa064ce9e82\") "
Sep 11 13:04:42.803905 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:42.803879    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "21d92bf5-4c7e-439f-ab2c-caa064ce9e82" (UID: "21d92bf5-4c7e-439f-ab2c-caa064ce9e82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:04:42.805823 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:42.805789    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-kube-api-access-jt4lm" (OuterVolumeSpecName: "kube-api-access-jt4lm") pod "21d92bf5-4c7e-439f-ab2c-caa064ce9e82" (UID: "21d92bf5-4c7e-439f-ab2c-caa064ce9e82"). InnerVolumeSpecName "kube-api-access-jt4lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:04:42.905186 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:42.905099    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt4lm\" (UniqueName: \"kubernetes.io/projected/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-kube-api-access-jt4lm\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:04:42.905186 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:42.905132    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d92bf5-4c7e-439f-ab2c-caa064ce9e82-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:04:43.448486 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:43.448451    2564 generic.go:356] "Generic (PLEG): container finished" podID="21d92bf5-4c7e-439f-ab2c-caa064ce9e82" containerID="a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b" exitCode=0
Sep 11 13:04:43.448968 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:43.448496    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"
Sep 11 13:04:43.448968 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:43.448533    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk" event={"ID":"21d92bf5-4c7e-439f-ab2c-caa064ce9e82","Type":"ContainerDied","Data":"a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b"}
Sep 11 13:04:43.448968 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:43.448621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk" event={"ID":"21d92bf5-4c7e-439f-ab2c-caa064ce9e82","Type":"ContainerDied","Data":"f1c4816ac46e41e4d08db59d5e8780d4ceae841baedf0a42c6423cda6974ce1d"}
Sep 11 13:04:43.448968 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:43.448679    2564 scope.go:117] "RemoveContainer" containerID="a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b"
Sep 11 13:04:43.459537 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:43.459518    2564 scope.go:117] "RemoveContainer" containerID="a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b"
Sep 11 13:04:43.459879 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:04:43.459861    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b\": container with ID starting with a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b not found: ID does not exist" containerID="a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b"
Sep 11 13:04:43.459957 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:43.459886    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b"} err="failed to get container status \"a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b\": rpc error: code = NotFound desc = could not find container \"a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b\": container with ID starting with a9097795f0173e1fa8e56052ff22477a187e969b749e6d2e7a9ffa416b7ea31b not found: ID does not exist"
Sep 11 13:04:43.473246 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:43.473216    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"]
Sep 11 13:04:43.479793 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:43.479766    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5f1b61-77f9dd9786-x9jzk"]
Sep 11 13:04:44.484915 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:44.484871    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21d92bf5-4c7e-439f-ab2c-caa064ce9e82" path="/var/lib/kubelet/pods/21d92bf5-4c7e-439f-ab2c-caa064ce9e82/volumes"
Sep 11 13:04:56.453836 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.453795    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"]
Sep 11 13:04:56.454485 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454458    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d92bf5-4c7e-439f-ab2c-caa064ce9e82" containerName="payload-pre-processing"
Sep 11 13:04:56.454485 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454485    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d92bf5-4c7e-439f-ab2c-caa064ce9e82" containerName="payload-pre-processing"
Sep 11 13:04:56.454750 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454505    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e3607da-1712-4af3-8e30-648fecb7a210" containerName="payload-processing"
Sep 11 13:04:56.454750 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454514    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e3607da-1712-4af3-8e30-648fecb7a210" containerName="payload-processing"
Sep 11 13:04:56.454750 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454522    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84d1b945-8c2a-42fe-b5b9-c85678f5f6ae" containerName="maas-api"
Sep 11 13:04:56.454750 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454530    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="84d1b945-8c2a-42fe-b5b9-c85678f5f6ae" containerName="maas-api"
Sep 11 13:04:56.454750 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454571    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c37ffae-9bd4-40a5-ad33-0eb82bb744e4" containerName="payload-pre-processing"
Sep 11 13:04:56.454750 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454579    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c37ffae-9bd4-40a5-ad33-0eb82bb744e4" containerName="payload-pre-processing"
Sep 11 13:04:56.454750 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454729    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e3607da-1712-4af3-8e30-648fecb7a210" containerName="payload-processing"
Sep 11 13:04:56.454750 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454742    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c37ffae-9bd4-40a5-ad33-0eb82bb744e4" containerName="payload-pre-processing"
Sep 11 13:04:56.454750 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454752    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="84d1b945-8c2a-42fe-b5b9-c85678f5f6ae" containerName="maas-api"
Sep 11 13:04:56.455220 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.454763    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d92bf5-4c7e-439f-ab2c-caa064ce9e82" containerName="payload-pre-processing"
Sep 11 13:04:56.459444 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.459420    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.463511 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.463483    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-25a0b392-serving-cert\""
Sep 11 13:04:56.485831 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.485798    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"]
Sep 11 13:04:56.534067 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.534029    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-maas-api-tls\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.534067 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.534072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-metrics-certs\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.534294 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.534092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4k4h\" (UniqueName: \"kubernetes.io/projected/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-kube-api-access-w4k4h\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.534294 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.534178    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-openshift-service-ca\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.634960 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.634922    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-maas-api-tls\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.634960 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.634971    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-metrics-certs\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.635247 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.634997    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4k4h\" (UniqueName: \"kubernetes.io/projected/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-kube-api-access-w4k4h\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.635247 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.635033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-openshift-service-ca\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.635247 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:04:56.635092    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-serving-cert: secret "maas-api-e2e-derive-25a0b392-serving-cert" not found
Sep 11 13:04:56.635247 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:04:56.635169    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-maas-api-tls podName:e27c82f9-bc00-4fca-b6a9-23a8796d2dc5 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:57.13514895 +0000 UTC m=+1065.197455908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-maas-api-tls") pod "maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d" (UID: "e27c82f9-bc00-4fca-b6a9-23a8796d2dc5") : secret "maas-api-e2e-derive-25a0b392-serving-cert" not found
Sep 11 13:04:56.635832 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.635808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-openshift-service-ca\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.637518 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.637499    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-metrics-certs\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:56.662733 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:56.662701    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4k4h\" (UniqueName: \"kubernetes.io/projected/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-kube-api-access-w4k4h\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:57.138711 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.138676    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-maas-api-tls\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:57.141970 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.141937    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-maas-api-tls\") pod \"maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:57.370246 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.370205    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:57.387027 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.386996    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"]
Sep 11 13:04:57.391045 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.390988    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:04:57.393746 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.393722    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-25a0b392\""
Sep 11 13:04:57.393941 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.393920    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-25a0b392-dockercfg-6ppw2\""
Sep 11 13:04:57.401941 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.401910    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"]
Sep 11 13:04:57.444713 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.443620    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36f8f607-d47a-4d77-a785-f981cdb4ce73-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4\" (UID: \"36f8f607-d47a-4d77-a785-f981cdb4ce73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:04:57.444713 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.443805    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpcrz\" (UniqueName: \"kubernetes.io/projected/36f8f607-d47a-4d77-a785-f981cdb4ce73-kube-api-access-dpcrz\") pod \"payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4\" (UID: \"36f8f607-d47a-4d77-a785-f981cdb4ce73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:04:57.545391 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.545346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpcrz\" (UniqueName: \"kubernetes.io/projected/36f8f607-d47a-4d77-a785-f981cdb4ce73-kube-api-access-dpcrz\") pod \"payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4\" (UID: \"36f8f607-d47a-4d77-a785-f981cdb4ce73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:04:57.545908 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.545487    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36f8f607-d47a-4d77-a785-f981cdb4ce73-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4\" (UID: \"36f8f607-d47a-4d77-a785-f981cdb4ce73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:04:57.546409 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.546381    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36f8f607-d47a-4d77-a785-f981cdb4ce73-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4\" (UID: \"36f8f607-d47a-4d77-a785-f981cdb4ce73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:04:57.569341 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.569304    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpcrz\" (UniqueName: \"kubernetes.io/projected/36f8f607-d47a-4d77-a785-f981cdb4ce73-kube-api-access-dpcrz\") pod \"payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4\" (UID: \"36f8f607-d47a-4d77-a785-f981cdb4ce73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:04:57.680128 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.680044    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"]
Sep 11 13:04:57.695851 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:04:57.695820    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode27c82f9_bc00_4fca_b6a9_23a8796d2dc5.slice/crio-abeddd30bc7514b6ac7ad91975c3f652d8870955d79ddf7df3904f6b511e8f94 WatchSource:0}: Error finding container abeddd30bc7514b6ac7ad91975c3f652d8870955d79ddf7df3904f6b511e8f94: Status 404 returned error can't find the container with id abeddd30bc7514b6ac7ad91975c3f652d8870955d79ddf7df3904f6b511e8f94
Sep 11 13:04:57.697246 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.697225    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:04:57.708661 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:57.708622    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:04:58.006589 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:58.006556    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"]
Sep 11 13:04:58.008419 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:04:58.008392    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f8f607_d47a_4d77_a785_f981cdb4ce73.slice/crio-137da826811de0281f5741969abccee94081f6760de05ef5ffd2b12008570b21 WatchSource:0}: Error finding container 137da826811de0281f5741969abccee94081f6760de05ef5ffd2b12008570b21: Status 404 returned error can't find the container with id 137da826811de0281f5741969abccee94081f6760de05ef5ffd2b12008570b21
Sep 11 13:04:58.538345 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:58.538305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4" event={"ID":"36f8f607-d47a-4d77-a785-f981cdb4ce73","Type":"ContainerStarted","Data":"ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870"}
Sep 11 13:04:58.538545 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:58.538363    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4" event={"ID":"36f8f607-d47a-4d77-a785-f981cdb4ce73","Type":"ContainerStarted","Data":"137da826811de0281f5741969abccee94081f6760de05ef5ffd2b12008570b21"}
Sep 11 13:04:58.538545 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:58.538395    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:04:58.539928 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:58.539900    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d" event={"ID":"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5","Type":"ContainerStarted","Data":"d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1"}
Sep 11 13:04:58.540114 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:58.539945    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d" event={"ID":"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5","Type":"ContainerStarted","Data":"abeddd30bc7514b6ac7ad91975c3f652d8870955d79ddf7df3904f6b511e8f94"}
Sep 11 13:04:58.540114 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:58.540004    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:04:58.558964 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:58.558912    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4" podStartSLOduration=1.5588933539999998 podStartE2EDuration="1.558893354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:58.557732359 +0000 UTC m=+1066.620039318" watchObservedRunningTime="2026-09-11 13:04:58.558893354 +0000 UTC m=+1066.621200319"
Sep 11 13:04:58.588189 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:04:58.588136    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d" podStartSLOduration=2.140812798 podStartE2EDuration="2.588117287s" totalImagesPullingTime="447.304489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:58.585859866 +0000 UTC m=+1066.648166850" watchObservedRunningTime="2026-09-11 13:04:58.588117287 +0000 UTC m=+1066.650424245"
Sep 11 13:05:03.739436 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.739397    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"]
Sep 11 13:05:03.745122 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.745091    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.748481 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.748452    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-3dec4b-serving-cert\""
Sep 11 13:05:03.763430 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.763301    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"]
Sep 11 13:05:03.808221 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.808179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-maas-api-tls\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.808424 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.808240    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-metrics-certs\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.808424 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.808368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-openshift-service-ca\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.808527 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.808437    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hskg7\" (UniqueName: \"kubernetes.io/projected/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-kube-api-access-hskg7\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.909792 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.909740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-maas-api-tls\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.910009 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.909815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-metrics-certs\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.910009 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.909868    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-openshift-service-ca\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.910009 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.909908    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hskg7\" (UniqueName: \"kubernetes.io/projected/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-kube-api-access-hskg7\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.910009 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:05:03.909922    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-serving-cert: secret "maas-api-e2e-explicit-3dec4b-serving-cert" not found
Sep 11 13:05:03.910009 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:05:03.909992    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-maas-api-tls podName:8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:04.409970568 +0000 UTC m=+1072.472277522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-maas-api-tls") pod "maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67" (UID: "8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7") : secret "maas-api-e2e-explicit-3dec4b-serving-cert" not found
Sep 11 13:05:03.910656 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.910621    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-openshift-service-ca\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.912394 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.912369    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-metrics-certs\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:03.923378 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:03.923347    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hskg7\" (UniqueName: \"kubernetes.io/projected/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-kube-api-access-hskg7\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:04.414927 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:04.414832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-maas-api-tls\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:04.417438 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:04.417403    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-maas-api-tls\") pod \"maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:04.553302 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:04.553273    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:05:04.657907 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:04.657871    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:04.936595 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:04.936569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"]
Sep 11 13:05:04.938723 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:05:04.938690    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9b0bfc_fb0f_4e8e_ada6_78a5030015e7.slice/crio-2d6d103af36d49b687ec136559907ddc7bc4526f568b1c94be3d7f7be2b29392 WatchSource:0}: Error finding container 2d6d103af36d49b687ec136559907ddc7bc4526f568b1c94be3d7f7be2b29392: Status 404 returned error can't find the container with id 2d6d103af36d49b687ec136559907ddc7bc4526f568b1c94be3d7f7be2b29392
Sep 11 13:05:05.590136 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:05.590101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67" event={"ID":"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7","Type":"ContainerStarted","Data":"2d6d103af36d49b687ec136559907ddc7bc4526f568b1c94be3d7f7be2b29392"}
Sep 11 13:05:06.596830 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:06.596789    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67" event={"ID":"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7","Type":"ContainerStarted","Data":"823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5"}
Sep 11 13:05:06.597221 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:06.596974    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:06.620528 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:06.620468    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67" podStartSLOduration=2.908137847 podStartE2EDuration="3.620450166s" totalImagesPullingTime="712.312319ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:06.620190784 +0000 UTC m=+1074.682497743" watchObservedRunningTime="2026-09-11 13:05:06.620450166 +0000 UTC m=+1074.682757123"
Sep 11 13:05:09.549280 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:09.549247    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:05:12.608893 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:12.608862    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:05:37.685726 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:37.685686    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"]
Sep 11 13:05:37.686366 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:37.685978    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d" podUID="e27c82f9-bc00-4fca-b6a9-23a8796d2dc5" containerName="maas-api" containerID="cri-o://d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1" gracePeriod=30
Sep 11 13:05:37.757926 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:37.757703    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"]
Sep 11 13:05:37.758197 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:37.757948    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4" podUID="36f8f607-d47a-4d77-a785-f981cdb4ce73" containerName="payload-pre-processing" containerID="cri-o://ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870" gracePeriod=30
Sep 11 13:05:38.021961 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.021928    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:05:38.059558 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.059526    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-openshift-service-ca\") pod \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") "
Sep 11 13:05:38.059758 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.059600    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-maas-api-tls\") pod \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") "
Sep 11 13:05:38.059840 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.059815    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-metrics-certs\") pod \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") "
Sep 11 13:05:38.059916 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.059903    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4k4h\" (UniqueName: \"kubernetes.io/projected/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-kube-api-access-w4k4h\") pod \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\" (UID: \"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5\") "
Sep 11 13:05:38.059970 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.059939    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e27c82f9-bc00-4fca-b6a9-23a8796d2dc5" (UID: "e27c82f9-bc00-4fca-b6a9-23a8796d2dc5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:05:38.060318 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.060272    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:05:38.062298 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.062265    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e27c82f9-bc00-4fca-b6a9-23a8796d2dc5" (UID: "e27c82f9-bc00-4fca-b6a9-23a8796d2dc5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:05:38.062298 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.062281    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e27c82f9-bc00-4fca-b6a9-23a8796d2dc5" (UID: "e27c82f9-bc00-4fca-b6a9-23a8796d2dc5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:05:38.062712 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.062689    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-kube-api-access-w4k4h" (OuterVolumeSpecName: "kube-api-access-w4k4h") pod "e27c82f9-bc00-4fca-b6a9-23a8796d2dc5" (UID: "e27c82f9-bc00-4fca-b6a9-23a8796d2dc5"). InnerVolumeSpecName "kube-api-access-w4k4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:05:38.076938 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.076899    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:05:38.160945 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.160911    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpcrz\" (UniqueName: \"kubernetes.io/projected/36f8f607-d47a-4d77-a785-f981cdb4ce73-kube-api-access-dpcrz\") pod \"36f8f607-d47a-4d77-a785-f981cdb4ce73\" (UID: \"36f8f607-d47a-4d77-a785-f981cdb4ce73\") "
Sep 11 13:05:38.161152 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.161024    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36f8f607-d47a-4d77-a785-f981cdb4ce73-plugins-config-volume\") pod \"36f8f607-d47a-4d77-a785-f981cdb4ce73\" (UID: \"36f8f607-d47a-4d77-a785-f981cdb4ce73\") "
Sep 11 13:05:38.161339 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.161309    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:05:38.161416 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.161343    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:05:38.161416 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.161358    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4k4h\" (UniqueName: \"kubernetes.io/projected/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5-kube-api-access-w4k4h\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:05:38.161933 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.161432    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36f8f607-d47a-4d77-a785-f981cdb4ce73-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "36f8f607-d47a-4d77-a785-f981cdb4ce73" (UID: "36f8f607-d47a-4d77-a785-f981cdb4ce73"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:05:38.163144 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.163118    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f8f607-d47a-4d77-a785-f981cdb4ce73-kube-api-access-dpcrz" (OuterVolumeSpecName: "kube-api-access-dpcrz") pod "36f8f607-d47a-4d77-a785-f981cdb4ce73" (UID: "36f8f607-d47a-4d77-a785-f981cdb4ce73"). InnerVolumeSpecName "kube-api-access-dpcrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:05:38.262209 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.262118    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpcrz\" (UniqueName: \"kubernetes.io/projected/36f8f607-d47a-4d77-a785-f981cdb4ce73-kube-api-access-dpcrz\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:05:38.262209 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.262152    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36f8f607-d47a-4d77-a785-f981cdb4ce73-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:05:38.789013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.788978    2564 generic.go:356] "Generic (PLEG): container finished" podID="36f8f607-d47a-4d77-a785-f981cdb4ce73" containerID="ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870" exitCode=0
Sep 11 13:05:38.789464 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.789023    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"
Sep 11 13:05:38.789464 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.789063    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4" event={"ID":"36f8f607-d47a-4d77-a785-f981cdb4ce73","Type":"ContainerDied","Data":"ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870"}
Sep 11 13:05:38.789464 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.789123    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4" event={"ID":"36f8f607-d47a-4d77-a785-f981cdb4ce73","Type":"ContainerDied","Data":"137da826811de0281f5741969abccee94081f6760de05ef5ffd2b12008570b21"}
Sep 11 13:05:38.789464 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.789156    2564 scope.go:117] "RemoveContainer" containerID="ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870"
Sep 11 13:05:38.790938 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.790855    2564 generic.go:356] "Generic (PLEG): container finished" podID="e27c82f9-bc00-4fca-b6a9-23a8796d2dc5" containerID="d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1" exitCode=0
Sep 11 13:05:38.790938 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.790931    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"
Sep 11 13:05:38.791115 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.790941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d" event={"ID":"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5","Type":"ContainerDied","Data":"d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1"}
Sep 11 13:05:38.791115 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.790997    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d" event={"ID":"e27c82f9-bc00-4fca-b6a9-23a8796d2dc5","Type":"ContainerDied","Data":"abeddd30bc7514b6ac7ad91975c3f652d8870955d79ddf7df3904f6b511e8f94"}
Sep 11 13:05:38.798861 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.798839    2564 scope.go:117] "RemoveContainer" containerID="ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870"
Sep 11 13:05:38.799223 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:05:38.799202    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870\": container with ID starting with ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870 not found: ID does not exist" containerID="ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870"
Sep 11 13:05:38.799306 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.799231    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870"} err="failed to get container status \"ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870\": rpc error: code = NotFound desc = could not find container \"ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870\": container with ID starting with ba39f8c65a31ecc9fedb0a1bca9518928b75ee3661128b9640e2e7bcc7705870 not found: ID does not exist"
Sep 11 13:05:38.799306 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.799251    2564 scope.go:117] "RemoveContainer" containerID="d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1"
Sep 11 13:05:38.809136 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.809108    2564 scope.go:117] "RemoveContainer" containerID="d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1"
Sep 11 13:05:38.809556 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:05:38.809536    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1\": container with ID starting with d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1 not found: ID does not exist" containerID="d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1"
Sep 11 13:05:38.809676 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.809566    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1"} err="failed to get container status \"d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1\": rpc error: code = NotFound desc = could not find container \"d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1\": container with ID starting with d81d156565292abb1ec7c0dc7174033dd49a28a45fdac4eab425580722f545b1 not found: ID does not exist"
Sep 11 13:05:38.835204 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.835166    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"]
Sep 11 13:05:38.844274 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.844238    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-25a0b392-5fd946dd6c-prr2d"]
Sep 11 13:05:38.886671 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.886613    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"]
Sep 11 13:05:38.893701 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:38.893635    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-25a0b392-764f5fc698-5g9w4"]
Sep 11 13:05:40.491044 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:40.491004    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36f8f607-d47a-4d77-a785-f981cdb4ce73" path="/var/lib/kubelet/pods/36f8f607-d47a-4d77-a785-f981cdb4ce73/volumes"
Sep 11 13:05:40.491554 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:05:40.491531    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e27c82f9-bc00-4fca-b6a9-23a8796d2dc5" path="/var/lib/kubelet/pods/e27c82f9-bc00-4fca-b6a9-23a8796d2dc5/volumes"
Sep 11 13:06:25.007359 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.007319    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"]
Sep 11 13:06:25.012123 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.011341    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67" podUID="8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7" containerName="maas-api" containerID="cri-o://823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5" gracePeriod=30
Sep 11 13:06:25.287515 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.287477    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:06:25.314915 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.314885    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-maas-api-tls\") pod \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") "
Sep 11 13:06:25.314915 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.314924    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-metrics-certs\") pod \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") "
Sep 11 13:06:25.315182 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.314973    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hskg7\" (UniqueName: \"kubernetes.io/projected/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-kube-api-access-hskg7\") pod \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") "
Sep 11 13:06:25.315182 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.315019    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-openshift-service-ca\") pod \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\" (UID: \"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7\") "
Sep 11 13:06:25.315490 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.315459    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7" (UID: "8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:25.317218 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.317183    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7" (UID: "8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:06:25.318065 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.317908    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-kube-api-access-hskg7" (OuterVolumeSpecName: "kube-api-access-hskg7") pod "8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7" (UID: "8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7"). InnerVolumeSpecName "kube-api-access-hskg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:06:25.318065 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.318044    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7" (UID: "8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:06:25.416763 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.416721    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:06:25.416763 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.416758    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:06:25.416763 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.416767    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:06:25.416763 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:25.416776    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hskg7\" (UniqueName: \"kubernetes.io/projected/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7-kube-api-access-hskg7\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:06:26.064341 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:26.064308    2564 generic.go:356] "Generic (PLEG): container finished" podID="8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7" containerID="823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5" exitCode=0
Sep 11 13:06:26.064795 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:26.064351    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"
Sep 11 13:06:26.064795 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:26.064363    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67" event={"ID":"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7","Type":"ContainerDied","Data":"823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5"}
Sep 11 13:06:26.064795 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:26.064411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67" event={"ID":"8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7","Type":"ContainerDied","Data":"2d6d103af36d49b687ec136559907ddc7bc4526f568b1c94be3d7f7be2b29392"}
Sep 11 13:06:26.064795 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:26.064435    2564 scope.go:117] "RemoveContainer" containerID="823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5"
Sep 11 13:06:26.074633 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:26.074607    2564 scope.go:117] "RemoveContainer" containerID="823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5"
Sep 11 13:06:26.074976 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:06:26.074953    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5\": container with ID starting with 823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5 not found: ID does not exist" containerID="823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5"
Sep 11 13:06:26.075060 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:26.074986    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5"} err="failed to get container status \"823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5\": rpc error: code = NotFound desc = could not find container \"823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5\": container with ID starting with 823eae1a931f31e4d44ee4af37b071feb88d8f70b9b9a8130dd6a5dbdf05bcb5 not found: ID does not exist"
Sep 11 13:06:26.086360 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:26.086321    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"]
Sep 11 13:06:26.092673 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:26.092623    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-3dec4b-76759cc9d7-bsc67"]
Sep 11 13:06:26.485221 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:06:26.485187    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7" path="/var/lib/kubelet/pods/8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7/volumes"
Sep 11 13:07:12.510870 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:12.510841    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:07:12.513504 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:12.513483    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:07:19.823287 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.823248    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"]
Sep 11 13:07:19.823755 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.823745    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f8f607-d47a-4d77-a785-f981cdb4ce73" containerName="payload-pre-processing"
Sep 11 13:07:19.823805 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.823759    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f8f607-d47a-4d77-a785-f981cdb4ce73" containerName="payload-pre-processing"
Sep 11 13:07:19.823805 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.823777    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e27c82f9-bc00-4fca-b6a9-23a8796d2dc5" containerName="maas-api"
Sep 11 13:07:19.823805 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.823782    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e27c82f9-bc00-4fca-b6a9-23a8796d2dc5" containerName="maas-api"
Sep 11 13:07:19.823805 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.823804    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7" containerName="maas-api"
Sep 11 13:07:19.823931 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.823810    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7" containerName="maas-api"
Sep 11 13:07:19.823931 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.823869    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e27c82f9-bc00-4fca-b6a9-23a8796d2dc5" containerName="maas-api"
Sep 11 13:07:19.823931 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.823879    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9b0bfc-fb0f-4e8e-ada6-78a5030015e7" containerName="maas-api"
Sep 11 13:07:19.823931 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.823886    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="36f8f607-d47a-4d77-a785-f981cdb4ce73" containerName="payload-pre-processing"
Sep 11 13:07:19.828229 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.828201    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:19.831619 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.831595    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f294ac01-serving-cert\""
Sep 11 13:07:19.850745 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.850705    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"]
Sep 11 13:07:19.939851 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.939806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-metrics-certs\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:19.940095 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.940023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npqtq\" (UniqueName: \"kubernetes.io/projected/4e1dad7b-76af-4758-836d-53d6da5216c1-kube-api-access-npqtq\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:19.940095 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.940074    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:19.940221 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:19.940130    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1dad7b-76af-4758-836d-53d6da5216c1-openshift-service-ca\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:20.041128 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:20.041076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npqtq\" (UniqueName: \"kubernetes.io/projected/4e1dad7b-76af-4758-836d-53d6da5216c1-kube-api-access-npqtq\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:20.041410 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:20.041144    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:20.041410 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:20.041186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1dad7b-76af-4758-836d-53d6da5216c1-openshift-service-ca\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:20.041410 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:20.041218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-metrics-certs\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:20.041410 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:07:20.041301    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-serving-cert: secret "maas-api-e2e-mt-f294ac01-serving-cert" not found
Sep 11 13:07:20.041410 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:07:20.041376    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls podName:4e1dad7b-76af-4758-836d-53d6da5216c1 nodeName:}" failed. No retries permitted until 2026-09-11 13:07:20.541352287 +0000 UTC m=+1208.603659237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls") pod "maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml" (UID: "4e1dad7b-76af-4758-836d-53d6da5216c1") : secret "maas-api-e2e-mt-f294ac01-serving-cert" not found
Sep 11 13:07:20.042034 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:20.042006    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1dad7b-76af-4758-836d-53d6da5216c1-openshift-service-ca\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:20.043766 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:20.043742    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-metrics-certs\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:20.050198 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:20.050169    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npqtq\" (UniqueName: \"kubernetes.io/projected/4e1dad7b-76af-4758-836d-53d6da5216c1-kube-api-access-npqtq\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:20.546356 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:20.546319    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:20.546586 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:07:20.546480    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-serving-cert: secret "maas-api-e2e-mt-f294ac01-serving-cert" not found
Sep 11 13:07:20.546586 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:07:20.546547    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls podName:4e1dad7b-76af-4758-836d-53d6da5216c1 nodeName:}" failed. No retries permitted until 2026-09-11 13:07:21.546531899 +0000 UTC m=+1209.608838840 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls") pod "maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml" (UID: "4e1dad7b-76af-4758-836d-53d6da5216c1") : secret "maas-api-e2e-mt-f294ac01-serving-cert" not found
Sep 11 13:07:21.557333 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:21.557295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:21.560208 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:21.560175    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls\") pod \"maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:21.649472 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:21.649425    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:21.944024 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:21.943995    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"]
Sep 11 13:07:21.945726 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:07:21.945698    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1dad7b_76af_4758_836d_53d6da5216c1.slice/crio-edf29791ad2bfe1f4a6f0387123d0de7501dd54b0f1be22e7c27e3db6415c175 WatchSource:0}: Error finding container edf29791ad2bfe1f4a6f0387123d0de7501dd54b0f1be22e7c27e3db6415c175: Status 404 returned error can't find the container with id edf29791ad2bfe1f4a6f0387123d0de7501dd54b0f1be22e7c27e3db6415c175
Sep 11 13:07:22.299584 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.299500    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"]
Sep 11 13:07:22.299872 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.299824    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p" podUID="0c84452e-2be8-4b4e-828c-b06abad45818" containerName="maas-api" containerID="cri-o://02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da" gracePeriod=30
Sep 11 13:07:22.379715 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.379673    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"]
Sep 11 13:07:22.379955 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.379927    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr" podUID="3494cc44-80ed-4561-b6be-02462497f361" containerName="payload-processing" containerID="cri-o://159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04" gracePeriod=30
Sep 11 13:07:22.385616 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.385517    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"]
Sep 11 13:07:22.385859 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.385777    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh" podUID="7c06f7d7-14ea-424b-a67e-2fbe560b5003" containerName="payload-pre-processing" containerID="cri-o://0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e" gracePeriod=30
Sep 11 13:07:22.394594 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.394552    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml" event={"ID":"4e1dad7b-76af-4758-836d-53d6da5216c1","Type":"ContainerStarted","Data":"edf29791ad2bfe1f4a6f0387123d0de7501dd54b0f1be22e7c27e3db6415c175"}
Sep 11 13:07:22.693948 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.693915    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:07:22.771432 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.771394    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-555xb\" (UniqueName: \"kubernetes.io/projected/3494cc44-80ed-4561-b6be-02462497f361-kube-api-access-555xb\") pod \"3494cc44-80ed-4561-b6be-02462497f361\" (UID: \"3494cc44-80ed-4561-b6be-02462497f361\") "
Sep 11 13:07:22.771628 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.771499    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3494cc44-80ed-4561-b6be-02462497f361-plugins-config-volume\") pod \"3494cc44-80ed-4561-b6be-02462497f361\" (UID: \"3494cc44-80ed-4561-b6be-02462497f361\") "
Sep 11 13:07:22.771933 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.771903    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3494cc44-80ed-4561-b6be-02462497f361-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3494cc44-80ed-4561-b6be-02462497f361" (UID: "3494cc44-80ed-4561-b6be-02462497f361"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:07:22.773560 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.773538    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3494cc44-80ed-4561-b6be-02462497f361-kube-api-access-555xb" (OuterVolumeSpecName: "kube-api-access-555xb") pod "3494cc44-80ed-4561-b6be-02462497f361" (UID: "3494cc44-80ed-4561-b6be-02462497f361"). InnerVolumeSpecName "kube-api-access-555xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:07:22.805896 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.805866    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:07:22.808777 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.808749    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:07:22.872944 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.872908    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-metrics-certs\") pod \"0c84452e-2be8-4b4e-828c-b06abad45818\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") "
Sep 11 13:07:22.873136 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.873120    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-maas-api-tls\") pod \"0c84452e-2be8-4b4e-828c-b06abad45818\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") "
Sep 11 13:07:22.873226 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.873152    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkfd2\" (UniqueName: \"kubernetes.io/projected/0c84452e-2be8-4b4e-828c-b06abad45818-kube-api-access-gkfd2\") pod \"0c84452e-2be8-4b4e-828c-b06abad45818\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") "
Sep 11 13:07:22.873226 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.873201    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c84452e-2be8-4b4e-828c-b06abad45818-openshift-service-ca\") pod \"0c84452e-2be8-4b4e-828c-b06abad45818\" (UID: \"0c84452e-2be8-4b4e-828c-b06abad45818\") "
Sep 11 13:07:22.873938 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.873494    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3494cc44-80ed-4561-b6be-02462497f361-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:22.873938 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.873523    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-555xb\" (UniqueName: \"kubernetes.io/projected/3494cc44-80ed-4561-b6be-02462497f361-kube-api-access-555xb\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:22.873938 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.873891    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c84452e-2be8-4b4e-828c-b06abad45818-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0c84452e-2be8-4b4e-828c-b06abad45818" (UID: "0c84452e-2be8-4b4e-828c-b06abad45818"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:07:22.875611 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.875571    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0c84452e-2be8-4b4e-828c-b06abad45818" (UID: "0c84452e-2be8-4b4e-828c-b06abad45818"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:07:22.875842 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.875808    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0c84452e-2be8-4b4e-828c-b06abad45818" (UID: "0c84452e-2be8-4b4e-828c-b06abad45818"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:07:22.875940 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.875874    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c84452e-2be8-4b4e-828c-b06abad45818-kube-api-access-gkfd2" (OuterVolumeSpecName: "kube-api-access-gkfd2") pod "0c84452e-2be8-4b4e-828c-b06abad45818" (UID: "0c84452e-2be8-4b4e-828c-b06abad45818"). InnerVolumeSpecName "kube-api-access-gkfd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:07:22.974304 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.974262    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9m56\" (UniqueName: \"kubernetes.io/projected/7c06f7d7-14ea-424b-a67e-2fbe560b5003-kube-api-access-n9m56\") pod \"7c06f7d7-14ea-424b-a67e-2fbe560b5003\" (UID: \"7c06f7d7-14ea-424b-a67e-2fbe560b5003\") "
Sep 11 13:07:22.974481 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.974351    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c06f7d7-14ea-424b-a67e-2fbe560b5003-plugins-config-volume\") pod \"7c06f7d7-14ea-424b-a67e-2fbe560b5003\" (UID: \"7c06f7d7-14ea-424b-a67e-2fbe560b5003\") "
Sep 11 13:07:22.974566 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.974554    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkfd2\" (UniqueName: \"kubernetes.io/projected/0c84452e-2be8-4b4e-828c-b06abad45818-kube-api-access-gkfd2\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:22.974600 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.974569    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c84452e-2be8-4b4e-828c-b06abad45818-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:22.974600 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.974578    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:22.974600 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.974586    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84452e-2be8-4b4e-828c-b06abad45818-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:22.974794 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.974765    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c06f7d7-14ea-424b-a67e-2fbe560b5003-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c06f7d7-14ea-424b-a67e-2fbe560b5003" (UID: "7c06f7d7-14ea-424b-a67e-2fbe560b5003"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:07:22.976480 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:22.976459    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c06f7d7-14ea-424b-a67e-2fbe560b5003-kube-api-access-n9m56" (OuterVolumeSpecName: "kube-api-access-n9m56") pod "7c06f7d7-14ea-424b-a67e-2fbe560b5003" (UID: "7c06f7d7-14ea-424b-a67e-2fbe560b5003"). InnerVolumeSpecName "kube-api-access-n9m56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:07:23.075552 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.075512    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c06f7d7-14ea-424b-a67e-2fbe560b5003-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:23.075552 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.075546    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9m56\" (UniqueName: \"kubernetes.io/projected/7c06f7d7-14ea-424b-a67e-2fbe560b5003-kube-api-access-n9m56\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:23.402524 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.402482    2564 generic.go:356] "Generic (PLEG): container finished" podID="7c06f7d7-14ea-424b-a67e-2fbe560b5003" containerID="0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e" exitCode=0
Sep 11 13:07:23.402759 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.402555    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"
Sep 11 13:07:23.402759 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.402566    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh" event={"ID":"7c06f7d7-14ea-424b-a67e-2fbe560b5003","Type":"ContainerDied","Data":"0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e"}
Sep 11 13:07:23.402759 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.402679    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh" event={"ID":"7c06f7d7-14ea-424b-a67e-2fbe560b5003","Type":"ContainerDied","Data":"fd5093d0eec02baf8c6bfc776b9064212c72a52817a7cd9735debd9008805726"}
Sep 11 13:07:23.402759 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.402711    2564 scope.go:117] "RemoveContainer" containerID="0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e"
Sep 11 13:07:23.404028 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.404005    2564 generic.go:356] "Generic (PLEG): container finished" podID="3494cc44-80ed-4561-b6be-02462497f361" containerID="159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04" exitCode=0
Sep 11 13:07:23.404119 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.404073    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"
Sep 11 13:07:23.404119 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.404082    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr" event={"ID":"3494cc44-80ed-4561-b6be-02462497f361","Type":"ContainerDied","Data":"159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04"}
Sep 11 13:07:23.404231 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.404134    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr" event={"ID":"3494cc44-80ed-4561-b6be-02462497f361","Type":"ContainerDied","Data":"97a227faeab5ea3a1674c10d707d189f27d54418cd9aa3f0532998c27c5ea07e"}
Sep 11 13:07:23.405779 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.405741    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c84452e-2be8-4b4e-828c-b06abad45818" containerID="02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da" exitCode=0
Sep 11 13:07:23.405918 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.405815    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"
Sep 11 13:07:23.405918 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.405831    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p" event={"ID":"0c84452e-2be8-4b4e-828c-b06abad45818","Type":"ContainerDied","Data":"02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da"}
Sep 11 13:07:23.406275 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.406249    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p" event={"ID":"0c84452e-2be8-4b4e-828c-b06abad45818","Type":"ContainerDied","Data":"438e137a8d05f6d9f28d1a8cffcd128ec2c4000c961d9e498ebbfc0419ca4f75"}
Sep 11 13:07:23.407819 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.407796    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml" event={"ID":"4e1dad7b-76af-4758-836d-53d6da5216c1","Type":"ContainerStarted","Data":"5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8"}
Sep 11 13:07:23.407974 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.407957    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:23.414337 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.414158    2564 scope.go:117] "RemoveContainer" containerID="0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e"
Sep 11 13:07:23.414555 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:07:23.414527    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e\": container with ID starting with 0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e not found: ID does not exist" containerID="0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e"
Sep 11 13:07:23.414622 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.414568    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e"} err="failed to get container status \"0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e\": rpc error: code = NotFound desc = could not find container \"0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e\": container with ID starting with 0fe307a13198aadf2b92b23fa4cc4983f1be74ea4e61eeab9a4c735cd6b6437e not found: ID does not exist"
Sep 11 13:07:23.414622 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.414592    2564 scope.go:117] "RemoveContainer" containerID="159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04"
Sep 11 13:07:23.424420 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.424401    2564 scope.go:117] "RemoveContainer" containerID="159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04"
Sep 11 13:07:23.424781 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:07:23.424763    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04\": container with ID starting with 159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04 not found: ID does not exist" containerID="159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04"
Sep 11 13:07:23.424880 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.424787    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04"} err="failed to get container status \"159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04\": rpc error: code = NotFound desc = could not find container \"159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04\": container with ID starting with 159cece83e949a5e2bc5fffaba497295beffe76698fa6a6db3769feeae3a0e04 not found: ID does not exist"
Sep 11 13:07:23.424880 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.424804    2564 scope.go:117] "RemoveContainer" containerID="02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da"
Sep 11 13:07:23.433578 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.433556    2564 scope.go:117] "RemoveContainer" containerID="02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da"
Sep 11 13:07:23.434061 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:07:23.434038    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da\": container with ID starting with 02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da not found: ID does not exist" containerID="02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da"
Sep 11 13:07:23.434140 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.434072    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da"} err="failed to get container status \"02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da\": rpc error: code = NotFound desc = could not find container \"02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da\": container with ID starting with 02110aa9b62e982a1e999a8261b0d7521d390c10c89fdd7e82f750b10a66e0da not found: ID does not exist"
Sep 11 13:07:23.439724 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.439673    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml" podStartSLOduration=3.681051129 podStartE2EDuration="4.439636615s" totalImagesPullingTime="758.585486ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:07:23.437947632 +0000 UTC m=+1211.500254612" watchObservedRunningTime="2026-09-11 13:07:23.439636615 +0000 UTC m=+1211.501943574"
Sep 11 13:07:23.453358 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.453324    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"]
Sep 11 13:07:23.461708 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.461675    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p"]
Sep 11 13:07:23.471860 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.471826    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"]
Sep 11 13:07:23.478063 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.478026    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0e2299-69b54d668c-pvsdr"]
Sep 11 13:07:23.490322 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.490290    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"]
Sep 11 13:07:23.495186 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:23.495156    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0e2299-85c8644fd-fwcvh"]
Sep 11 13:07:24.484308 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:24.484268    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c84452e-2be8-4b4e-828c-b06abad45818" path="/var/lib/kubelet/pods/0c84452e-2be8-4b4e-828c-b06abad45818/volumes"
Sep 11 13:07:24.484850 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:24.484829    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3494cc44-80ed-4561-b6be-02462497f361" path="/var/lib/kubelet/pods/3494cc44-80ed-4561-b6be-02462497f361/volumes"
Sep 11 13:07:24.485321 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:24.485301    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c06f7d7-14ea-424b-a67e-2fbe560b5003" path="/var/lib/kubelet/pods/7c06f7d7-14ea-424b-a67e-2fbe560b5003/volumes"
Sep 11 13:07:25.761302 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:25.761253    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0e2299-5fcdbb7bb8-zlb2p" podUID="0c84452e-2be8-4b4e-828c-b06abad45818" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.49:8443/health\": context deadline exceeded"
Sep 11 13:07:29.422181 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:29.422153    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:07:40.157976 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.157930    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"]
Sep 11 13:07:40.158499 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.158205    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8" podUID="9ddbfdb8-89fe-4950-83ae-639fde042751" containerName="maas-api" containerID="cri-o://f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec" gracePeriod=30
Sep 11 13:07:40.294771 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.294729    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"]
Sep 11 13:07:40.295014 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.294964    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t" podUID="dbdb2ef1-7705-430d-8e86-32ccb52a4def" containerName="payload-processing" containerID="cri-o://6c68e3abf5a4d754be9f53de71b2b33e58f499e0d6cb369248ca95276997b5dd" gracePeriod=30
Sep 11 13:07:40.439113 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.439083    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:07:40.510051 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.509985    2564 generic.go:356] "Generic (PLEG): container finished" podID="9ddbfdb8-89fe-4950-83ae-639fde042751" containerID="f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec" exitCode=0
Sep 11 13:07:40.510051 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.510022    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8" event={"ID":"9ddbfdb8-89fe-4950-83ae-639fde042751","Type":"ContainerDied","Data":"f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec"}
Sep 11 13:07:40.510562 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.510534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8" event={"ID":"9ddbfdb8-89fe-4950-83ae-639fde042751","Type":"ContainerDied","Data":"dc73b0fb357c831fc4e89d6f5fd7ee7b3264c0a35d79d02b668a12aef9b79606"}
Sep 11 13:07:40.510739 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.510583    2564 scope.go:117] "RemoveContainer" containerID="f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec"
Sep 11 13:07:40.510739 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.510678    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"
Sep 11 13:07:40.512339 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.512285    2564 generic.go:356] "Generic (PLEG): container finished" podID="dbdb2ef1-7705-430d-8e86-32ccb52a4def" containerID="6c68e3abf5a4d754be9f53de71b2b33e58f499e0d6cb369248ca95276997b5dd" exitCode=0
Sep 11 13:07:40.512446 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.512413    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t" event={"ID":"dbdb2ef1-7705-430d-8e86-32ccb52a4def","Type":"ContainerDied","Data":"6c68e3abf5a4d754be9f53de71b2b33e58f499e0d6cb369248ca95276997b5dd"}
Sep 11 13:07:40.520538 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.520518    2564 scope.go:117] "RemoveContainer" containerID="f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec"
Sep 11 13:07:40.520986 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:07:40.520961    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec\": container with ID starting with f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec not found: ID does not exist" containerID="f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec"
Sep 11 13:07:40.521104 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.520994    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec"} err="failed to get container status \"f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec\": rpc error: code = NotFound desc = could not find container \"f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec\": container with ID starting with f13fc0e80a5d7517037f395348aa5891dde29e3b759904ccb68068ea20e1e2ec not found: ID does not exist"
Sep 11 13:07:40.548609 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.548583    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-metrics-certs\") pod \"9ddbfdb8-89fe-4950-83ae-639fde042751\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") "
Sep 11 13:07:40.548770 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.548632    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgfjp\" (UniqueName: \"kubernetes.io/projected/9ddbfdb8-89fe-4950-83ae-639fde042751-kube-api-access-lgfjp\") pod \"9ddbfdb8-89fe-4950-83ae-639fde042751\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") "
Sep 11 13:07:40.548770 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.548762    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-maas-api-tls\") pod \"9ddbfdb8-89fe-4950-83ae-639fde042751\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") "
Sep 11 13:07:40.548885 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.548823    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ddbfdb8-89fe-4950-83ae-639fde042751-openshift-service-ca\") pod \"9ddbfdb8-89fe-4950-83ae-639fde042751\" (UID: \"9ddbfdb8-89fe-4950-83ae-639fde042751\") "
Sep 11 13:07:40.549386 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.549357    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ddbfdb8-89fe-4950-83ae-639fde042751-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ddbfdb8-89fe-4950-83ae-639fde042751" (UID: "9ddbfdb8-89fe-4950-83ae-639fde042751"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:07:40.551269 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.551239    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9ddbfdb8-89fe-4950-83ae-639fde042751" (UID: "9ddbfdb8-89fe-4950-83ae-639fde042751"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:07:40.551448 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.551426    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ddbfdb8-89fe-4950-83ae-639fde042751-kube-api-access-lgfjp" (OuterVolumeSpecName: "kube-api-access-lgfjp") pod "9ddbfdb8-89fe-4950-83ae-639fde042751" (UID: "9ddbfdb8-89fe-4950-83ae-639fde042751"). InnerVolumeSpecName "kube-api-access-lgfjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:07:40.551545 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.551519    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ddbfdb8-89fe-4950-83ae-639fde042751" (UID: "9ddbfdb8-89fe-4950-83ae-639fde042751"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:07:40.559470 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.559445    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:07:40.650145 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.650103    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkzfb\" (UniqueName: \"kubernetes.io/projected/dbdb2ef1-7705-430d-8e86-32ccb52a4def-kube-api-access-wkzfb\") pod \"dbdb2ef1-7705-430d-8e86-32ccb52a4def\" (UID: \"dbdb2ef1-7705-430d-8e86-32ccb52a4def\") "
Sep 11 13:07:40.650298 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.650163    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbdb2ef1-7705-430d-8e86-32ccb52a4def-plugins-config-volume\") pod \"dbdb2ef1-7705-430d-8e86-32ccb52a4def\" (UID: \"dbdb2ef1-7705-430d-8e86-32ccb52a4def\") "
Sep 11 13:07:40.650405 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.650394    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:40.650451 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.650409    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ddbfdb8-89fe-4950-83ae-639fde042751-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:40.650451 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.650420    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ddbfdb8-89fe-4950-83ae-639fde042751-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:40.650451 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.650428    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgfjp\" (UniqueName: \"kubernetes.io/projected/9ddbfdb8-89fe-4950-83ae-639fde042751-kube-api-access-lgfjp\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:40.650686 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.650621    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbdb2ef1-7705-430d-8e86-32ccb52a4def-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dbdb2ef1-7705-430d-8e86-32ccb52a4def" (UID: "dbdb2ef1-7705-430d-8e86-32ccb52a4def"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:07:40.652229 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.652202    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbdb2ef1-7705-430d-8e86-32ccb52a4def-kube-api-access-wkzfb" (OuterVolumeSpecName: "kube-api-access-wkzfb") pod "dbdb2ef1-7705-430d-8e86-32ccb52a4def" (UID: "dbdb2ef1-7705-430d-8e86-32ccb52a4def"). InnerVolumeSpecName "kube-api-access-wkzfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:07:40.751099 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.751021    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbdb2ef1-7705-430d-8e86-32ccb52a4def-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:40.751099 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.751048    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkzfb\" (UniqueName: \"kubernetes.io/projected/dbdb2ef1-7705-430d-8e86-32ccb52a4def-kube-api-access-wkzfb\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:07:40.832998 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.832961    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"]
Sep 11 13:07:40.839878 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:40.839840    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de3842-5d84c84496-9vwv8"]
Sep 11 13:07:41.520594 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:41.520507    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"
Sep 11 13:07:41.520594 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:41.520528    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t" event={"ID":"dbdb2ef1-7705-430d-8e86-32ccb52a4def","Type":"ContainerDied","Data":"1d367b17d0bda58e5b917e6814b2c210fd7b4f8ae0acc1532daa49aa3ab1b738"}
Sep 11 13:07:41.520594 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:41.520584    2564 scope.go:117] "RemoveContainer" containerID="6c68e3abf5a4d754be9f53de71b2b33e58f499e0d6cb369248ca95276997b5dd"
Sep 11 13:07:41.551087 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:41.551057    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"]
Sep 11 13:07:41.555196 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:41.555172    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-de3842-6845849f87-qcn2t"]
Sep 11 13:07:42.484941 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:42.484890    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ddbfdb8-89fe-4950-83ae-639fde042751" path="/var/lib/kubelet/pods/9ddbfdb8-89fe-4950-83ae-639fde042751/volumes"
Sep 11 13:07:42.485512 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:42.485489    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbdb2ef1-7705-430d-8e86-32ccb52a4def" path="/var/lib/kubelet/pods/dbdb2ef1-7705-430d-8e86-32ccb52a4def/volumes"
Sep 11 13:07:54.618862 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.618824    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"]
Sep 11 13:07:54.619336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619232    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c06f7d7-14ea-424b-a67e-2fbe560b5003" containerName="payload-pre-processing"
Sep 11 13:07:54.619336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619244    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c06f7d7-14ea-424b-a67e-2fbe560b5003" containerName="payload-pre-processing"
Sep 11 13:07:54.619336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619253    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c84452e-2be8-4b4e-828c-b06abad45818" containerName="maas-api"
Sep 11 13:07:54.619336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619259    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c84452e-2be8-4b4e-828c-b06abad45818" containerName="maas-api"
Sep 11 13:07:54.619336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619266    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ddbfdb8-89fe-4950-83ae-639fde042751" containerName="maas-api"
Sep 11 13:07:54.619336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619272    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ddbfdb8-89fe-4950-83ae-639fde042751" containerName="maas-api"
Sep 11 13:07:54.619336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619280    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbdb2ef1-7705-430d-8e86-32ccb52a4def" containerName="payload-processing"
Sep 11 13:07:54.619336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619285    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbdb2ef1-7705-430d-8e86-32ccb52a4def" containerName="payload-processing"
Sep 11 13:07:54.619336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619306    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3494cc44-80ed-4561-b6be-02462497f361" containerName="payload-processing"
Sep 11 13:07:54.619336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619311    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3494cc44-80ed-4561-b6be-02462497f361" containerName="payload-processing"
Sep 11 13:07:54.619655 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619383    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3494cc44-80ed-4561-b6be-02462497f361" containerName="payload-processing"
Sep 11 13:07:54.619655 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619391    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ddbfdb8-89fe-4950-83ae-639fde042751" containerName="maas-api"
Sep 11 13:07:54.619655 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619401    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c06f7d7-14ea-424b-a67e-2fbe560b5003" containerName="payload-pre-processing"
Sep 11 13:07:54.619655 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619408    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c84452e-2be8-4b4e-828c-b06abad45818" containerName="maas-api"
Sep 11 13:07:54.619655 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.619414    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbdb2ef1-7705-430d-8e86-32ccb52a4def" containerName="payload-processing"
Sep 11 13:07:54.624060 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.624033    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.628904 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.628872    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-d4b0b9-serving-cert\""
Sep 11 13:07:54.649449 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.649412    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"]
Sep 11 13:07:54.686414 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.686373    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssc2t\" (UniqueName: \"kubernetes.io/projected/39695453-9282-4f0f-9279-83c1b1070a9a-kube-api-access-ssc2t\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.686619 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.686439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.686619 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.686527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39695453-9282-4f0f-9279-83c1b1070a9a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.686795 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.686636    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.788092 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.788053    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssc2t\" (UniqueName: \"kubernetes.io/projected/39695453-9282-4f0f-9279-83c1b1070a9a-kube-api-access-ssc2t\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.788291 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.788115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.788291 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.788143    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39695453-9282-4f0f-9279-83c1b1070a9a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.788291 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.788213    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.788971 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.788943    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39695453-9282-4f0f-9279-83c1b1070a9a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.790889 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.790862    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.790889 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.790886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.798208 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.798171    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssc2t\" (UniqueName: \"kubernetes.io/projected/39695453-9282-4f0f-9279-83c1b1070a9a-kube-api-access-ssc2t\") pod \"maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:54.936008 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:54.935913    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:55.247989 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.247962    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"]
Sep 11 13:07:55.250816 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:07:55.250788    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39695453_9282_4f0f_9279_83c1b1070a9a.slice/crio-4aafcc901666f53fd429f81d4bd6bdfd452c3451b9de94909d293508de360bf1 WatchSource:0}: Error finding container 4aafcc901666f53fd429f81d4bd6bdfd452c3451b9de94909d293508de360bf1: Status 404 returned error can't find the container with id 4aafcc901666f53fd429f81d4bd6bdfd452c3451b9de94909d293508de360bf1
Sep 11 13:07:55.472271 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.472222    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"]
Sep 11 13:07:55.479214 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.479184    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:07:55.482532 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.482502    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-d4b0b9-dockercfg-b26bq\""
Sep 11 13:07:55.482744 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.482503    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-d4b0b9\""
Sep 11 13:07:55.484359 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.484332    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"]
Sep 11 13:07:55.595374 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.595339    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhmhf\" (UniqueName: \"kubernetes.io/projected/9dc7aa67-7c68-4251-a339-cc5aa8a69792-kube-api-access-qhmhf\") pod \"payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd\" (UID: \"9dc7aa67-7c68-4251-a339-cc5aa8a69792\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:07:55.595612 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.595590    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dc7aa67-7c68-4251-a339-cc5aa8a69792-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd\" (UID: \"9dc7aa67-7c68-4251-a339-cc5aa8a69792\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:07:55.597867 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.597839    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq" event={"ID":"39695453-9282-4f0f-9279-83c1b1070a9a","Type":"ContainerStarted","Data":"4aafcc901666f53fd429f81d4bd6bdfd452c3451b9de94909d293508de360bf1"}
Sep 11 13:07:55.696536 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.696496    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dc7aa67-7c68-4251-a339-cc5aa8a69792-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd\" (UID: \"9dc7aa67-7c68-4251-a339-cc5aa8a69792\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:07:55.696946 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.696586    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhmhf\" (UniqueName: \"kubernetes.io/projected/9dc7aa67-7c68-4251-a339-cc5aa8a69792-kube-api-access-qhmhf\") pod \"payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd\" (UID: \"9dc7aa67-7c68-4251-a339-cc5aa8a69792\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:07:55.697274 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.697252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dc7aa67-7c68-4251-a339-cc5aa8a69792-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd\" (UID: \"9dc7aa67-7c68-4251-a339-cc5aa8a69792\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:07:55.706167 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.706133    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhmhf\" (UniqueName: \"kubernetes.io/projected/9dc7aa67-7c68-4251-a339-cc5aa8a69792-kube-api-access-qhmhf\") pod \"payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd\" (UID: \"9dc7aa67-7c68-4251-a339-cc5aa8a69792\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:07:55.792886 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:55.792852    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:07:56.075846 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:56.075814    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"]
Sep 11 13:07:56.078324 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:07:56.078286    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dc7aa67_7c68_4251_a339_cc5aa8a69792.slice/crio-4682c6c50cc40dd6aa5bd5e3bf92baeb294a463eea4c3d3fda7765749ac209a1 WatchSource:0}: Error finding container 4682c6c50cc40dd6aa5bd5e3bf92baeb294a463eea4c3d3fda7765749ac209a1: Status 404 returned error can't find the container with id 4682c6c50cc40dd6aa5bd5e3bf92baeb294a463eea4c3d3fda7765749ac209a1
Sep 11 13:07:56.605175 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:56.605138    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq" event={"ID":"39695453-9282-4f0f-9279-83c1b1070a9a","Type":"ContainerStarted","Data":"88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067"}
Sep 11 13:07:56.605387 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:56.605262    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:07:56.606761 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:56.606729    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd" event={"ID":"9dc7aa67-7c68-4251-a339-cc5aa8a69792","Type":"ContainerStarted","Data":"df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903"}
Sep 11 13:07:56.606903 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:56.606767    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd" event={"ID":"9dc7aa67-7c68-4251-a339-cc5aa8a69792","Type":"ContainerStarted","Data":"4682c6c50cc40dd6aa5bd5e3bf92baeb294a463eea4c3d3fda7765749ac209a1"}
Sep 11 13:07:56.606903 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:56.606830    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:07:56.631735 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:56.631685    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq" podStartSLOduration=2.258203125 podStartE2EDuration="2.631666455s" totalImagesPullingTime="373.46333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:07:56.629404528 +0000 UTC m=+1244.691711487" watchObservedRunningTime="2026-09-11 13:07:56.631666455 +0000 UTC m=+1244.693973409"
Sep 11 13:07:56.651897 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:07:56.651846    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd" podStartSLOduration=1.65183121 podStartE2EDuration="1.65183121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:07:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:07:56.649346181 +0000 UTC m=+1244.711653139" watchObservedRunningTime="2026-09-11 13:07:56.65183121 +0000 UTC m=+1244.714138167"
Sep 11 13:08:02.617746 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:02.617713    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:08:03.681836 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:03.681798    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"]
Sep 11 13:08:03.682279 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:03.682048    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml" podUID="4e1dad7b-76af-4758-836d-53d6da5216c1" containerName="maas-api" containerID="cri-o://5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8" gracePeriod=30
Sep 11 13:08:03.956296 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:03.956265    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:08:04.074346 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.074312    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npqtq\" (UniqueName: \"kubernetes.io/projected/4e1dad7b-76af-4758-836d-53d6da5216c1-kube-api-access-npqtq\") pod \"4e1dad7b-76af-4758-836d-53d6da5216c1\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") "
Sep 11 13:08:04.074551 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.074438    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1dad7b-76af-4758-836d-53d6da5216c1-openshift-service-ca\") pod \"4e1dad7b-76af-4758-836d-53d6da5216c1\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") "
Sep 11 13:08:04.074551 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.074482    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls\") pod \"4e1dad7b-76af-4758-836d-53d6da5216c1\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") "
Sep 11 13:08:04.074551 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.074513    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-metrics-certs\") pod \"4e1dad7b-76af-4758-836d-53d6da5216c1\" (UID: \"4e1dad7b-76af-4758-836d-53d6da5216c1\") "
Sep 11 13:08:04.074925 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.074893    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e1dad7b-76af-4758-836d-53d6da5216c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4e1dad7b-76af-4758-836d-53d6da5216c1" (UID: "4e1dad7b-76af-4758-836d-53d6da5216c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:08:04.076833 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.076793    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4e1dad7b-76af-4758-836d-53d6da5216c1" (UID: "4e1dad7b-76af-4758-836d-53d6da5216c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:08:04.076975 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.076852    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4e1dad7b-76af-4758-836d-53d6da5216c1" (UID: "4e1dad7b-76af-4758-836d-53d6da5216c1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:08:04.076975 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.076907    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e1dad7b-76af-4758-836d-53d6da5216c1-kube-api-access-npqtq" (OuterVolumeSpecName: "kube-api-access-npqtq") pod "4e1dad7b-76af-4758-836d-53d6da5216c1" (UID: "4e1dad7b-76af-4758-836d-53d6da5216c1"). InnerVolumeSpecName "kube-api-access-npqtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:08:04.175911 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.175871    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1dad7b-76af-4758-836d-53d6da5216c1-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:08:04.175911 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.175903    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:08:04.175911 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.175914    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1dad7b-76af-4758-836d-53d6da5216c1-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:08:04.176157 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.175924    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npqtq\" (UniqueName: \"kubernetes.io/projected/4e1dad7b-76af-4758-836d-53d6da5216c1-kube-api-access-npqtq\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:08:04.653114 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.653077    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e1dad7b-76af-4758-836d-53d6da5216c1" containerID="5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8" exitCode=0
Sep 11 13:08:04.653296 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.653131    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml" event={"ID":"4e1dad7b-76af-4758-836d-53d6da5216c1","Type":"ContainerDied","Data":"5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8"}
Sep 11 13:08:04.653296 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.653139    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"
Sep 11 13:08:04.653296 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.653172    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml" event={"ID":"4e1dad7b-76af-4758-836d-53d6da5216c1","Type":"ContainerDied","Data":"edf29791ad2bfe1f4a6f0387123d0de7501dd54b0f1be22e7c27e3db6415c175"}
Sep 11 13:08:04.653296 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.653193    2564 scope.go:117] "RemoveContainer" containerID="5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8"
Sep 11 13:08:04.663499 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.663478    2564 scope.go:117] "RemoveContainer" containerID="5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8"
Sep 11 13:08:04.663918 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:08:04.663896    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8\": container with ID starting with 5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8 not found: ID does not exist" containerID="5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8"
Sep 11 13:08:04.663987 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.663929    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8"} err="failed to get container status \"5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8\": rpc error: code = NotFound desc = could not find container \"5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8\": container with ID starting with 5cf6d9b07a01500054b5b0f0bb61664a28bcbb52a28739278b0659c8a77c48f8 not found: ID does not exist"
Sep 11 13:08:04.673242 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.673208    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"]
Sep 11 13:08:04.675128 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:04.675104    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f294ac01-59f6d8c667-9wqml"]
Sep 11 13:08:06.483836 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:06.483800    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e1dad7b-76af-4758-836d-53d6da5216c1" path="/var/lib/kubelet/pods/4e1dad7b-76af-4758-836d-53d6da5216c1/volumes"
Sep 11 13:08:07.614196 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:07.614165    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:08:23.446664 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.446600    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"]
Sep 11 13:08:23.447132 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.447062    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e1dad7b-76af-4758-836d-53d6da5216c1" containerName="maas-api"
Sep 11 13:08:23.447132 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.447077    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e1dad7b-76af-4758-836d-53d6da5216c1" containerName="maas-api"
Sep 11 13:08:23.447257 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.447155    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e1dad7b-76af-4758-836d-53d6da5216c1" containerName="maas-api"
Sep 11 13:08:23.451613 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.451589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.454317 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.454293    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-2e7f38-serving-cert\""
Sep 11 13:08:23.478049 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.478017    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"]
Sep 11 13:08:23.561583 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.561545    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.561829 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.561603    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c5caac5-88ad-480a-95d9-ee33cfc354f0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.561930 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.561907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.561987 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.561974    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vd4f\" (UniqueName: \"kubernetes.io/projected/4c5caac5-88ad-480a-95d9-ee33cfc354f0-kube-api-access-6vd4f\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.663255 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.663217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.663480 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.663275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vd4f\" (UniqueName: \"kubernetes.io/projected/4c5caac5-88ad-480a-95d9-ee33cfc354f0-kube-api-access-6vd4f\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.663480 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.663295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.663480 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.663337    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c5caac5-88ad-480a-95d9-ee33cfc354f0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.663480 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:08:23.663408    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-serving-cert: secret "maas-api-e2e-shared-b-w3-2e7f38-serving-cert" not found
Sep 11 13:08:23.663732 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:08:23.663500    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-maas-api-tls podName:4c5caac5-88ad-480a-95d9-ee33cfc354f0 nodeName:}" failed. No retries permitted until 2026-09-11 13:08:24.163477834 +0000 UTC m=+1272.225784789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-maas-api-tls") pod "maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb" (UID: "4c5caac5-88ad-480a-95d9-ee33cfc354f0") : secret "maas-api-e2e-shared-b-w3-2e7f38-serving-cert" not found
Sep 11 13:08:23.664117 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.664090    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c5caac5-88ad-480a-95d9-ee33cfc354f0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.665983 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.665960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:23.674032 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:23.674009    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vd4f\" (UniqueName: \"kubernetes.io/projected/4c5caac5-88ad-480a-95d9-ee33cfc354f0-kube-api-access-6vd4f\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:24.168860 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:24.168810    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:24.171246 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:24.171221    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:24.362622 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:24.362583    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:24.714745 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:24.714717    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"]
Sep 11 13:08:24.716413 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:08:24.716382    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c5caac5_88ad_480a_95d9_ee33cfc354f0.slice/crio-d1379a736728762c32d74d3f296e7a2bd66a854f8e68347bd0eb266dff537978 WatchSource:0}: Error finding container d1379a736728762c32d74d3f296e7a2bd66a854f8e68347bd0eb266dff537978: Status 404 returned error can't find the container with id d1379a736728762c32d74d3f296e7a2bd66a854f8e68347bd0eb266dff537978
Sep 11 13:08:24.770694 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:24.770658    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb" event={"ID":"4c5caac5-88ad-480a-95d9-ee33cfc354f0","Type":"ContainerStarted","Data":"d1379a736728762c32d74d3f296e7a2bd66a854f8e68347bd0eb266dff537978"}
Sep 11 13:08:25.777624 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:25.777586    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb" event={"ID":"4c5caac5-88ad-480a-95d9-ee33cfc354f0","Type":"ContainerStarted","Data":"3d7fecc43808ad05194b68519c5793a7640ee262bb9027c618941353be0cd33c"}
Sep 11 13:08:25.778128 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:25.777684    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:25.817563 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:25.817506    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb" podStartSLOduration=2.3705392339999998 podStartE2EDuration="2.817486889s" totalImagesPullingTime="446.947655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:08:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:08:25.817137217 +0000 UTC m=+1273.879444187" watchObservedRunningTime="2026-09-11 13:08:25.817486889 +0000 UTC m=+1273.879793899"
Sep 11 13:08:31.788813 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:31.788784    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:08:33.249423 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.249391    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"]
Sep 11 13:08:33.254688 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.254664    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.258408 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:08:33.258372    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" podUID="059ca660-b6e8-4c7d-b429-530e37f6c863" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.258534 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:08:33.258418    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-shared-a-w2-b81d9e-serving-cert\" is forbidden: User \"system:node:ip-10-0-136-55.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-136-55.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-b81d9e-serving-cert\"" type="*v1.Secret"
Sep 11 13:08:33.298460 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.298425    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"]
Sep 11 13:08:33.369238 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.369200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.369452 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.369255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q2vq\" (UniqueName: \"kubernetes.io/projected/059ca660-b6e8-4c7d-b429-530e37f6c863-kube-api-access-7q2vq\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.369452 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.369279    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.369452 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.369417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/059ca660-b6e8-4c7d-b429-530e37f6c863-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.470347 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.470303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.470553 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.470372    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q2vq\" (UniqueName: \"kubernetes.io/projected/059ca660-b6e8-4c7d-b429-530e37f6c863-kube-api-access-7q2vq\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.470553 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.470413    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.470553 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.470459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/059ca660-b6e8-4c7d-b429-530e37f6c863-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.471101 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.471080    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/059ca660-b6e8-4c7d-b429-530e37f6c863-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.472917 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.472893    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:33.480404 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:33.480374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q2vq\" (UniqueName: \"kubernetes.io/projected/059ca660-b6e8-4c7d-b429-530e37f6c863-kube-api-access-7q2vq\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:34.161524 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.161493    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-b81d9e-serving-cert\""
Sep 11 13:08:34.173571 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.173536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:34.300267 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.300227    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"]
Sep 11 13:08:34.304071 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.304049    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:08:34.319169 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.319147    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-b81d9e\""
Sep 11 13:08:34.319297 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.319147    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-b81d9e-dockercfg-w7qjn\""
Sep 11 13:08:34.337894 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.337857    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"]
Sep 11 13:08:34.377999 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.377958    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8\" (UID: \"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:08:34.378204 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.378152    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wsw6\" (UniqueName: \"kubernetes.io/projected/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-kube-api-access-9wsw6\") pod \"payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8\" (UID: \"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:08:34.464702 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.464567    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:34.479163 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.479122    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8\" (UID: \"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:08:34.479335 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.479318    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wsw6\" (UniqueName: \"kubernetes.io/projected/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-kube-api-access-9wsw6\") pod \"payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8\" (UID: \"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:08:34.479982 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.479957    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8\" (UID: \"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:08:34.489264 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.489234    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wsw6\" (UniqueName: \"kubernetes.io/projected/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-kube-api-access-9wsw6\") pod \"payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8\" (UID: \"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:08:34.615370 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.615337    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:08:34.745394 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.745366    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"]
Sep 11 13:08:34.748480 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:08:34.748448    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059ca660_b6e8_4c7d_b429_530e37f6c863.slice/crio-a0e6ff35992c852e3ebf213ea6578d2bf2466eb13ba20cdd2cd5b343fe28ca00 WatchSource:0}: Error finding container a0e6ff35992c852e3ebf213ea6578d2bf2466eb13ba20cdd2cd5b343fe28ca00: Status 404 returned error can't find the container with id a0e6ff35992c852e3ebf213ea6578d2bf2466eb13ba20cdd2cd5b343fe28ca00
Sep 11 13:08:34.828291 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.828251    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr" event={"ID":"059ca660-b6e8-4c7d-b429-530e37f6c863","Type":"ContainerStarted","Data":"a0e6ff35992c852e3ebf213ea6578d2bf2466eb13ba20cdd2cd5b343fe28ca00"}
Sep 11 13:08:34.898512 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:34.898484    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"]
Sep 11 13:08:34.901111 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:08:34.901083    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod653d8a01_2f0e_4fdf_9d7b_cae0f796c4af.slice/crio-20ebc951c779f4010aa048baf48613fcfe325aefef8b42450591b09336e7f2b9 WatchSource:0}: Error finding container 20ebc951c779f4010aa048baf48613fcfe325aefef8b42450591b09336e7f2b9: Status 404 returned error can't find the container with id 20ebc951c779f4010aa048baf48613fcfe325aefef8b42450591b09336e7f2b9
Sep 11 13:08:35.835560 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:35.835521    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8" event={"ID":"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af","Type":"ContainerStarted","Data":"f0b53e5034f3fb2430d7b2f4a91844caebdbd65f2678a9a90a15a3de830a0932"}
Sep 11 13:08:35.836071 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:35.835589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8" event={"ID":"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af","Type":"ContainerStarted","Data":"20ebc951c779f4010aa048baf48613fcfe325aefef8b42450591b09336e7f2b9"}
Sep 11 13:08:35.836071 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:35.835624    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:08:35.837241 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:35.837214    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr" event={"ID":"059ca660-b6e8-4c7d-b429-530e37f6c863","Type":"ContainerStarted","Data":"624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd"}
Sep 11 13:08:35.837322 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:35.837276    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:35.857509 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:35.857454    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8" podStartSLOduration=1.8574345 podStartE2EDuration="1.8574345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:08:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:08:35.855134579 +0000 UTC m=+1283.917441539" watchObservedRunningTime="2026-09-11 13:08:35.8574345 +0000 UTC m=+1283.919741457"
Sep 11 13:08:35.878720 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:35.878667    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr" podStartSLOduration=2.49980958 podStartE2EDuration="2.878634659s" totalImagesPullingTime="378.825079ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:08:35.877882452 +0000 UTC m=+1283.940189410" watchObservedRunningTime="2026-09-11 13:08:35.878634659 +0000 UTC m=+1283.940941617"
Sep 11 13:08:41.849379 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:41.849347    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:08:46.845401 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:08:46.845359    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:09:02.215504 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.215464    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"]
Sep 11 13:09:02.222419 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.222389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.225864 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.225831    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-7c542a-serving-cert\""
Sep 11 13:09:02.230920 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.230893    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"]
Sep 11 13:09:02.239831 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.239791    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.240105 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.239848    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.240105 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.239882    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbhvz\" (UniqueName: \"kubernetes.io/projected/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-kube-api-access-xbhvz\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.240105 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.239947    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.341146 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.341099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.341439 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.341416    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.341563 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.341549    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbhvz\" (UniqueName: \"kubernetes.io/projected/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-kube-api-access-xbhvz\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.341742 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.341726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.342042 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:09:02.341860    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-serving-cert: secret "maas-api-e2e-shared-b-w2-7c542a-serving-cert" not found
Sep 11 13:09:02.342042 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:09:02.341950    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-maas-api-tls podName:24c88e64-5ea8-4f2d-93b5-56ede95aaa83 nodeName:}" failed. No retries permitted until 2026-09-11 13:09:02.841929083 +0000 UTC m=+1310.904236023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-maas-api-tls") pod "maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2" (UID: "24c88e64-5ea8-4f2d-93b5-56ede95aaa83") : secret "maas-api-e2e-shared-b-w2-7c542a-serving-cert" not found
Sep 11 13:09:02.342376 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.342349    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.344039 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.344013    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.355090 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.355062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbhvz\" (UniqueName: \"kubernetes.io/projected/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-kube-api-access-xbhvz\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.848099 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.848057    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:02.850802 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:02.850771    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:03.136506 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:03.136412    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:03.423088 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:03.423051    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"]
Sep 11 13:09:03.425410 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:09:03.425374    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c88e64_5ea8_4f2d_93b5_56ede95aaa83.slice/crio-788482868dbf7a4e096126b0561b2b8bfa75804eb3d2d220a210d5ad995fedc8 WatchSource:0}: Error finding container 788482868dbf7a4e096126b0561b2b8bfa75804eb3d2d220a210d5ad995fedc8: Status 404 returned error can't find the container with id 788482868dbf7a4e096126b0561b2b8bfa75804eb3d2d220a210d5ad995fedc8
Sep 11 13:09:03.990854 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:03.990807    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2" event={"ID":"24c88e64-5ea8-4f2d-93b5-56ede95aaa83","Type":"ContainerStarted","Data":"403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4"}
Sep 11 13:09:03.991028 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:03.990877    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2" event={"ID":"24c88e64-5ea8-4f2d-93b5-56ede95aaa83","Type":"ContainerStarted","Data":"788482868dbf7a4e096126b0561b2b8bfa75804eb3d2d220a210d5ad995fedc8"}
Sep 11 13:09:03.991028 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:03.990905    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:10.001437 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:10.001403    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:09:10.027064 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:10.027000    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2" podStartSLOduration=7.689483041 podStartE2EDuration="8.026980423s" totalImagesPullingTime="337.497382ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:09:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:09:04.012022239 +0000 UTC m=+1312.074329198" watchObservedRunningTime="2026-09-11 13:09:10.026980423 +0000 UTC m=+1318.089287398"
Sep 11 13:09:42.546778 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.546747    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"]
Sep 11 13:09:42.550438 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.550419    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.552910 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.552889    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f27fde90-serving-cert\""
Sep 11 13:09:42.561115 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.561089    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"]
Sep 11 13:09:42.733406 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.733364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/604dd260-01a8-462d-8152-5f16087bddc5-openshift-service-ca\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.733406 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.733403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-maas-api-tls\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.733669 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.733506    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-metrics-certs\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.733669 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.733559    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l6bs\" (UniqueName: \"kubernetes.io/projected/604dd260-01a8-462d-8152-5f16087bddc5-kube-api-access-4l6bs\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.834912 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.834877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l6bs\" (UniqueName: \"kubernetes.io/projected/604dd260-01a8-462d-8152-5f16087bddc5-kube-api-access-4l6bs\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.835080 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.834970    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/604dd260-01a8-462d-8152-5f16087bddc5-openshift-service-ca\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.835080 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.834999    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-maas-api-tls\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.835080 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.835051    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-metrics-certs\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.835727 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.835681    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/604dd260-01a8-462d-8152-5f16087bddc5-openshift-service-ca\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.837574 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.837554    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-maas-api-tls\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.837743 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.837683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-metrics-certs\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.847682 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.847635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l6bs\" (UniqueName: \"kubernetes.io/projected/604dd260-01a8-462d-8152-5f16087bddc5-kube-api-access-4l6bs\") pod \"maas-api-e2e-mt-f27fde90-67c4499876-6m5xm\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:42.862554 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:42.862516    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:43.141479 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:43.141447    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"]
Sep 11 13:09:43.144955 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:09:43.144920    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod604dd260_01a8_462d_8152_5f16087bddc5.slice/crio-01b2af9c70ea2cb28c9ea50e2bda1febfd84784349a75679cf5392467ca6caa8 WatchSource:0}: Error finding container 01b2af9c70ea2cb28c9ea50e2bda1febfd84784349a75679cf5392467ca6caa8: Status 404 returned error can't find the container with id 01b2af9c70ea2cb28c9ea50e2bda1febfd84784349a75679cf5392467ca6caa8
Sep 11 13:09:43.213475 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:43.213427    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm" event={"ID":"604dd260-01a8-462d-8152-5f16087bddc5","Type":"ContainerStarted","Data":"01b2af9c70ea2cb28c9ea50e2bda1febfd84784349a75679cf5392467ca6caa8"}
Sep 11 13:09:44.221024 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:44.220983    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm" event={"ID":"604dd260-01a8-462d-8152-5f16087bddc5","Type":"ContainerStarted","Data":"289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72"}
Sep 11 13:09:44.221482 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:44.221099    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:50.232602 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:50.232570    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:09:50.254446 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:09:50.254385    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm" podStartSLOduration=7.83513405 podStartE2EDuration="8.254368286s" totalImagesPullingTime="419.234236ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:09:44.249394901 +0000 UTC m=+1352.311701872" watchObservedRunningTime="2026-09-11 13:09:50.254368286 +0000 UTC m=+1358.316675243"
Sep 11 13:11:01.742715 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:01.742673    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"]
Sep 11 13:11:01.743276 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:01.742875    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm" podUID="604dd260-01a8-462d-8152-5f16087bddc5" containerName="maas-api" containerID="cri-o://289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72" gracePeriod=30
Sep 11 13:11:02.005422 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.005357    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:11:02.025535 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.025497    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l6bs\" (UniqueName: \"kubernetes.io/projected/604dd260-01a8-462d-8152-5f16087bddc5-kube-api-access-4l6bs\") pod \"604dd260-01a8-462d-8152-5f16087bddc5\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") "
Sep 11 13:11:02.025793 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.025560    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-metrics-certs\") pod \"604dd260-01a8-462d-8152-5f16087bddc5\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") "
Sep 11 13:11:02.025793 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.025597    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/604dd260-01a8-462d-8152-5f16087bddc5-openshift-service-ca\") pod \"604dd260-01a8-462d-8152-5f16087bddc5\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") "
Sep 11 13:11:02.025793 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.025760    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-maas-api-tls\") pod \"604dd260-01a8-462d-8152-5f16087bddc5\" (UID: \"604dd260-01a8-462d-8152-5f16087bddc5\") "
Sep 11 13:11:02.026024 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.026001    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/604dd260-01a8-462d-8152-5f16087bddc5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "604dd260-01a8-462d-8152-5f16087bddc5" (UID: "604dd260-01a8-462d-8152-5f16087bddc5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:11:02.026234 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.026171    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/604dd260-01a8-462d-8152-5f16087bddc5-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:11:02.027866 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.027838    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "604dd260-01a8-462d-8152-5f16087bddc5" (UID: "604dd260-01a8-462d-8152-5f16087bddc5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:11:02.027976 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.027884    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "604dd260-01a8-462d-8152-5f16087bddc5" (UID: "604dd260-01a8-462d-8152-5f16087bddc5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:11:02.028040 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.027997    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/604dd260-01a8-462d-8152-5f16087bddc5-kube-api-access-4l6bs" (OuterVolumeSpecName: "kube-api-access-4l6bs") pod "604dd260-01a8-462d-8152-5f16087bddc5" (UID: "604dd260-01a8-462d-8152-5f16087bddc5"). InnerVolumeSpecName "kube-api-access-4l6bs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:11:02.127343 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.127303    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l6bs\" (UniqueName: \"kubernetes.io/projected/604dd260-01a8-462d-8152-5f16087bddc5-kube-api-access-4l6bs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:11:02.127343 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.127342    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:11:02.127562 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.127355    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/604dd260-01a8-462d-8152-5f16087bddc5-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:11:02.668371 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.668334    2564 generic.go:356] "Generic (PLEG): container finished" podID="604dd260-01a8-462d-8152-5f16087bddc5" containerID="289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72" exitCode=0
Sep 11 13:11:02.668572 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.668394    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"
Sep 11 13:11:02.668572 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.668423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm" event={"ID":"604dd260-01a8-462d-8152-5f16087bddc5","Type":"ContainerDied","Data":"289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72"}
Sep 11 13:11:02.668572 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.668490    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm" event={"ID":"604dd260-01a8-462d-8152-5f16087bddc5","Type":"ContainerDied","Data":"01b2af9c70ea2cb28c9ea50e2bda1febfd84784349a75679cf5392467ca6caa8"}
Sep 11 13:11:02.668572 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.668521    2564 scope.go:117] "RemoveContainer" containerID="289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72"
Sep 11 13:11:02.677673 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.677652    2564 scope.go:117] "RemoveContainer" containerID="289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72"
Sep 11 13:11:02.678027 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:11:02.678005    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72\": container with ID starting with 289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72 not found: ID does not exist" containerID="289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72"
Sep 11 13:11:02.678091 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.678037    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72"} err="failed to get container status \"289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72\": rpc error: code = NotFound desc = could not find container \"289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72\": container with ID starting with 289987dbc66dd393cc80cc10eb808f279c23a5b84ba6a97e534d31dc6aa68f72 not found: ID does not exist"
Sep 11 13:11:02.686385 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.686357    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"]
Sep 11 13:11:02.690357 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:02.690331    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f27fde90-67c4499876-6m5xm"]
Sep 11 13:11:04.484670 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:11:04.484570    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="604dd260-01a8-462d-8152-5f16087bddc5" path="/var/lib/kubelet/pods/604dd260-01a8-462d-8152-5f16087bddc5/volumes"
Sep 11 13:12:12.547363 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:12.547333    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:12:12.552528 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:12.552501    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:12:53.626581 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.626537    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"]
Sep 11 13:12:53.629123 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.627042    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="604dd260-01a8-462d-8152-5f16087bddc5" containerName="maas-api"
Sep 11 13:12:53.629123 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.627056    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="604dd260-01a8-462d-8152-5f16087bddc5" containerName="maas-api"
Sep 11 13:12:53.629123 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.627137    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="604dd260-01a8-462d-8152-5f16087bddc5" containerName="maas-api"
Sep 11 13:12:53.630443 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.630418    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.633199 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.633167    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-940f81-serving-cert\""
Sep 11 13:12:53.642523 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.642492    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"]
Sep 11 13:12:53.732786 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.732739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba879cbb-9f17-4ad7-bef0-e5b03c903627-openshift-service-ca\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.732967 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.732847    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-maas-api-tls\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.732967 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.732914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh9m8\" (UniqueName: \"kubernetes.io/projected/ba879cbb-9f17-4ad7-bef0-e5b03c903627-kube-api-access-jh9m8\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.732967 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.732960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-metrics-certs\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.833674 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.833600    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-maas-api-tls\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.833674 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.833662    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh9m8\" (UniqueName: \"kubernetes.io/projected/ba879cbb-9f17-4ad7-bef0-e5b03c903627-kube-api-access-jh9m8\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.833904 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.833701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-metrics-certs\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.833904 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.833735    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba879cbb-9f17-4ad7-bef0-e5b03c903627-openshift-service-ca\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.833904 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:12:53.833787    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-serving-cert: secret "maas-api-e2e-api-a-940f81-serving-cert" not found
Sep 11 13:12:53.833904 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:12:53.833861    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-maas-api-tls podName:ba879cbb-9f17-4ad7-bef0-e5b03c903627 nodeName:}" failed. No retries permitted until 2026-09-11 13:12:54.333839498 +0000 UTC m=+1542.396146436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-maas-api-tls") pod "maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb" (UID: "ba879cbb-9f17-4ad7-bef0-e5b03c903627") : secret "maas-api-e2e-api-a-940f81-serving-cert" not found
Sep 11 13:12:53.834351 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.834331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba879cbb-9f17-4ad7-bef0-e5b03c903627-openshift-service-ca\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.836409 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.836381    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-metrics-certs\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:53.845935 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:53.845901    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh9m8\" (UniqueName: \"kubernetes.io/projected/ba879cbb-9f17-4ad7-bef0-e5b03c903627-kube-api-access-jh9m8\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:54.339552 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.339514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-maas-api-tls\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:54.342634 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.342597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-maas-api-tls\") pod \"maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:54.444324 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.444284    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"]
Sep 11 13:12:54.449025 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.449000    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:12:54.451757 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.451729    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-940f81-dockercfg-gl6d7\""
Sep 11 13:12:54.451894 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.451734    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-940f81\""
Sep 11 13:12:54.456896 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.456871    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"]
Sep 11 13:12:54.541500 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.541461    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx5mw\" (UniqueName: \"kubernetes.io/projected/b9095fd0-5447-457c-995f-490ae08411fb-kube-api-access-qx5mw\") pod \"payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f\" (UID: \"b9095fd0-5447-457c-995f-490ae08411fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:12:54.541743 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.541615    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9095fd0-5447-457c-995f-490ae08411fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f\" (UID: \"b9095fd0-5447-457c-995f-490ae08411fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:12:54.544474 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.544446    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:54.643979 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.642831    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx5mw\" (UniqueName: \"kubernetes.io/projected/b9095fd0-5447-457c-995f-490ae08411fb-kube-api-access-qx5mw\") pod \"payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f\" (UID: \"b9095fd0-5447-457c-995f-490ae08411fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:12:54.643979 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.642984    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9095fd0-5447-457c-995f-490ae08411fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f\" (UID: \"b9095fd0-5447-457c-995f-490ae08411fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:12:54.643979 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.643924    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9095fd0-5447-457c-995f-490ae08411fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f\" (UID: \"b9095fd0-5447-457c-995f-490ae08411fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:12:54.653098 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.653065    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx5mw\" (UniqueName: \"kubernetes.io/projected/b9095fd0-5447-457c-995f-490ae08411fb-kube-api-access-qx5mw\") pod \"payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f\" (UID: \"b9095fd0-5447-457c-995f-490ae08411fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:12:54.762930 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.762880    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:12:54.831535 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.831458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"]
Sep 11 13:12:54.835271 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:12:54.835235    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba879cbb_9f17_4ad7_bef0_e5b03c903627.slice/crio-f32586a80bdf180df603e4dd4152541341cf7dd60e5388fb79768e5b94024058 WatchSource:0}: Error finding container f32586a80bdf180df603e4dd4152541341cf7dd60e5388fb79768e5b94024058: Status 404 returned error can't find the container with id f32586a80bdf180df603e4dd4152541341cf7dd60e5388fb79768e5b94024058
Sep 11 13:12:54.837295 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:54.837272    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:12:55.041064 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:55.041011    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"]
Sep 11 13:12:55.042860 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:12:55.042828    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9095fd0_5447_457c_995f_490ae08411fb.slice/crio-fc01c09c7fa95295b74d9bd39681d645fc1d9f77793de556e74dcffaeb61882a WatchSource:0}: Error finding container fc01c09c7fa95295b74d9bd39681d645fc1d9f77793de556e74dcffaeb61882a: Status 404 returned error can't find the container with id fc01c09c7fa95295b74d9bd39681d645fc1d9f77793de556e74dcffaeb61882a
Sep 11 13:12:55.296731 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:55.296611    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f" event={"ID":"b9095fd0-5447-457c-995f-490ae08411fb","Type":"ContainerStarted","Data":"1cb226180bcb38e649006c1c5e23d6e0136613d07b0030160e05404217cba727"}
Sep 11 13:12:55.296731 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:55.296706    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f" event={"ID":"b9095fd0-5447-457c-995f-490ae08411fb","Type":"ContainerStarted","Data":"fc01c09c7fa95295b74d9bd39681d645fc1d9f77793de556e74dcffaeb61882a"}
Sep 11 13:12:55.296978 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:55.296736    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:12:55.297933 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:55.297910    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb" event={"ID":"ba879cbb-9f17-4ad7-bef0-e5b03c903627","Type":"ContainerStarted","Data":"f32586a80bdf180df603e4dd4152541341cf7dd60e5388fb79768e5b94024058"}
Sep 11 13:12:55.319400 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:55.319339    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f" podStartSLOduration=1.319317554 podStartE2EDuration="1.319317554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:12:55.317526111 +0000 UTC m=+1543.379833069" watchObservedRunningTime="2026-09-11 13:12:55.319317554 +0000 UTC m=+1543.381624503"
Sep 11 13:12:56.305250 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:56.305212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb" event={"ID":"ba879cbb-9f17-4ad7-bef0-e5b03c903627","Type":"ContainerStarted","Data":"ae447c6aef062c114dfacd4f255262898343d08e6197d377913c78078d64d01c"}
Sep 11 13:12:56.305673 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:56.305405    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:12:56.326294 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:12:56.326243    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb" podStartSLOduration=2.831831673 podStartE2EDuration="3.326225289s" totalImagesPullingTime="494.393616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:12:56.323518273 +0000 UTC m=+1544.385825232" watchObservedRunningTime="2026-09-11 13:12:56.326225289 +0000 UTC m=+1544.388532247"
Sep 11 13:13:02.315864 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:02.315836    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:13:06.308159 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:06.308127    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:13:23.527980 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.527936    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"]
Sep 11 13:13:23.531844 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.531819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:13:23.534242 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.534217    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-ccae06\""
Sep 11 13:13:23.534355 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.534225    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-ccae06-dockercfg-vk8gs\""
Sep 11 13:13:23.546702 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.545424    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"]
Sep 11 13:13:23.633365 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.633330    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"]
Sep 11 13:13:23.638057 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.638024    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:13:23.638550 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.638520    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2wvz\" (UniqueName: \"kubernetes.io/projected/fc3049f1-f1d1-446a-b820-06f250ec9616-kube-api-access-s2wvz\") pod \"payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck\" (UID: \"fc3049f1-f1d1-446a-b820-06f250ec9616\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:13:23.638888 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.638862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc3049f1-f1d1-446a-b820-06f250ec9616-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck\" (UID: \"fc3049f1-f1d1-446a-b820-06f250ec9616\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:13:23.648193 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.648168    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"]
Sep 11 13:13:23.740305 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.740268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtjh2\" (UniqueName: \"kubernetes.io/projected/f3d97fcb-4261-4ef7-9c21-970b53082f4c-kube-api-access-gtjh2\") pod \"payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t\" (UID: \"f3d97fcb-4261-4ef7-9c21-970b53082f4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:13:23.740505 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.740356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2wvz\" (UniqueName: \"kubernetes.io/projected/fc3049f1-f1d1-446a-b820-06f250ec9616-kube-api-access-s2wvz\") pod \"payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck\" (UID: \"fc3049f1-f1d1-446a-b820-06f250ec9616\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:13:23.740505 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.740387    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d97fcb-4261-4ef7-9c21-970b53082f4c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t\" (UID: \"f3d97fcb-4261-4ef7-9c21-970b53082f4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:13:23.740631 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.740542    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc3049f1-f1d1-446a-b820-06f250ec9616-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck\" (UID: \"fc3049f1-f1d1-446a-b820-06f250ec9616\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:13:23.741477 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.741452    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc3049f1-f1d1-446a-b820-06f250ec9616-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck\" (UID: \"fc3049f1-f1d1-446a-b820-06f250ec9616\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:13:23.750556 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.750519    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2wvz\" (UniqueName: \"kubernetes.io/projected/fc3049f1-f1d1-446a-b820-06f250ec9616-kube-api-access-s2wvz\") pod \"payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck\" (UID: \"fc3049f1-f1d1-446a-b820-06f250ec9616\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:13:23.842020 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.841986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d97fcb-4261-4ef7-9c21-970b53082f4c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t\" (UID: \"f3d97fcb-4261-4ef7-9c21-970b53082f4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:13:23.842233 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.842082    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtjh2\" (UniqueName: \"kubernetes.io/projected/f3d97fcb-4261-4ef7-9c21-970b53082f4c-kube-api-access-gtjh2\") pod \"payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t\" (UID: \"f3d97fcb-4261-4ef7-9c21-970b53082f4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:13:23.842766 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.842741    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d97fcb-4261-4ef7-9c21-970b53082f4c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t\" (UID: \"f3d97fcb-4261-4ef7-9c21-970b53082f4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:13:23.844876 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.844854    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:13:23.850463 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.850439    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtjh2\" (UniqueName: \"kubernetes.io/projected/f3d97fcb-4261-4ef7-9c21-970b53082f4c-kube-api-access-gtjh2\") pod \"payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t\" (UID: \"f3d97fcb-4261-4ef7-9c21-970b53082f4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:13:23.954507 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:23.954473    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:13:24.119133 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:24.119100    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"]
Sep 11 13:13:24.120599 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:13:24.120572    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3049f1_f1d1_446a_b820_06f250ec9616.slice/crio-573c4a339a214fa0bba0dbdc6a04f7de788d82cfccbbd395e450b66bd47cf6dc WatchSource:0}: Error finding container 573c4a339a214fa0bba0dbdc6a04f7de788d82cfccbbd395e450b66bd47cf6dc: Status 404 returned error can't find the container with id 573c4a339a214fa0bba0dbdc6a04f7de788d82cfccbbd395e450b66bd47cf6dc
Sep 11 13:13:24.232913 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:24.232890    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"]
Sep 11 13:13:24.234479 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:13:24.234450    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d97fcb_4261_4ef7_9c21_970b53082f4c.slice/crio-8bfbcc01c78e010ff35de41fe6ec7063e01d40639293e91b8b802031d214ebea WatchSource:0}: Error finding container 8bfbcc01c78e010ff35de41fe6ec7063e01d40639293e91b8b802031d214ebea: Status 404 returned error can't find the container with id 8bfbcc01c78e010ff35de41fe6ec7063e01d40639293e91b8b802031d214ebea
Sep 11 13:13:24.465307 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:24.465213    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t" event={"ID":"f3d97fcb-4261-4ef7-9c21-970b53082f4c","Type":"ContainerStarted","Data":"6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715"}
Sep 11 13:13:24.465307 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:24.465285    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t" event={"ID":"f3d97fcb-4261-4ef7-9c21-970b53082f4c","Type":"ContainerStarted","Data":"8bfbcc01c78e010ff35de41fe6ec7063e01d40639293e91b8b802031d214ebea"}
Sep 11 13:13:24.465563 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:24.465316    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:13:24.467374 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:24.467348    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck" event={"ID":"fc3049f1-f1d1-446a-b820-06f250ec9616","Type":"ContainerStarted","Data":"7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd"}
Sep 11 13:13:24.467512 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:24.467396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck" event={"ID":"fc3049f1-f1d1-446a-b820-06f250ec9616","Type":"ContainerStarted","Data":"573c4a339a214fa0bba0dbdc6a04f7de788d82cfccbbd395e450b66bd47cf6dc"}
Sep 11 13:13:24.467512 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:24.467426    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:13:24.485849 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:24.485787    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t" podStartSLOduration=1.485767257 podStartE2EDuration="1.485767257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:13:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:13:24.482713639 +0000 UTC m=+1572.545020600" watchObservedRunningTime="2026-09-11 13:13:24.485767257 +0000 UTC m=+1572.548074212"
Sep 11 13:13:24.501966 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:24.501914    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck" podStartSLOduration=1.501892786 podStartE2EDuration="1.501892786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:13:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:13:24.500961063 +0000 UTC m=+1572.563268022" watchObservedRunningTime="2026-09-11 13:13:24.501892786 +0000 UTC m=+1572.564199745"
Sep 11 13:13:35.475123 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:35.475086    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:13:35.475728 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:13:35.475176    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:14:05.498587 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.498387    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"]
Sep 11 13:14:05.499103 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.498633    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb" podUID="ba879cbb-9f17-4ad7-bef0-e5b03c903627" containerName="maas-api" containerID="cri-o://ae447c6aef062c114dfacd4f255262898343d08e6197d377913c78078d64d01c" gracePeriod=30
Sep 11 13:14:05.564913 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.564863    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"]
Sep 11 13:14:05.565401 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.565370    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f" podUID="b9095fd0-5447-457c-995f-490ae08411fb" containerName="payload-pre-processing" containerID="cri-o://1cb226180bcb38e649006c1c5e23d6e0136613d07b0030160e05404217cba727" gracePeriod=30
Sep 11 13:14:05.709568 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.709534    2564 generic.go:356] "Generic (PLEG): container finished" podID="b9095fd0-5447-457c-995f-490ae08411fb" containerID="1cb226180bcb38e649006c1c5e23d6e0136613d07b0030160e05404217cba727" exitCode=0
Sep 11 13:14:05.709817 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.709605    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f" event={"ID":"b9095fd0-5447-457c-995f-490ae08411fb","Type":"ContainerDied","Data":"1cb226180bcb38e649006c1c5e23d6e0136613d07b0030160e05404217cba727"}
Sep 11 13:14:05.711564 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.711422    2564 generic.go:356] "Generic (PLEG): container finished" podID="ba879cbb-9f17-4ad7-bef0-e5b03c903627" containerID="ae447c6aef062c114dfacd4f255262898343d08e6197d377913c78078d64d01c" exitCode=0
Sep 11 13:14:05.711564 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.711537    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb" event={"ID":"ba879cbb-9f17-4ad7-bef0-e5b03c903627","Type":"ContainerDied","Data":"ae447c6aef062c114dfacd4f255262898343d08e6197d377913c78078d64d01c"}
Sep 11 13:14:05.827687 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.827659    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:14:05.869312 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.869280    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-metrics-certs\") pod \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") "
Sep 11 13:14:05.869462 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.869362    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jh9m8\" (UniqueName: \"kubernetes.io/projected/ba879cbb-9f17-4ad7-bef0-e5b03c903627-kube-api-access-jh9m8\") pod \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") "
Sep 11 13:14:05.869462 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.869431    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba879cbb-9f17-4ad7-bef0-e5b03c903627-openshift-service-ca\") pod \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") "
Sep 11 13:14:05.869462 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.869458    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-maas-api-tls\") pod \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\" (UID: \"ba879cbb-9f17-4ad7-bef0-e5b03c903627\") "
Sep 11 13:14:05.869877 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.869843    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba879cbb-9f17-4ad7-bef0-e5b03c903627-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ba879cbb-9f17-4ad7-bef0-e5b03c903627" (UID: "ba879cbb-9f17-4ad7-bef0-e5b03c903627"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:14:05.870233 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.870212    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:14:05.871768 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.871729    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ba879cbb-9f17-4ad7-bef0-e5b03c903627" (UID: "ba879cbb-9f17-4ad7-bef0-e5b03c903627"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:14:05.872054 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.872033    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ba879cbb-9f17-4ad7-bef0-e5b03c903627" (UID: "ba879cbb-9f17-4ad7-bef0-e5b03c903627"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:14:05.872118 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.872064    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba879cbb-9f17-4ad7-bef0-e5b03c903627-kube-api-access-jh9m8" (OuterVolumeSpecName: "kube-api-access-jh9m8") pod "ba879cbb-9f17-4ad7-bef0-e5b03c903627" (UID: "ba879cbb-9f17-4ad7-bef0-e5b03c903627"). InnerVolumeSpecName "kube-api-access-jh9m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:14:05.970436 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.970399    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9095fd0-5447-457c-995f-490ae08411fb-plugins-config-volume\") pod \"b9095fd0-5447-457c-995f-490ae08411fb\" (UID: \"b9095fd0-5447-457c-995f-490ae08411fb\") "
Sep 11 13:14:05.970636 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.970471    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx5mw\" (UniqueName: \"kubernetes.io/projected/b9095fd0-5447-457c-995f-490ae08411fb-kube-api-access-qx5mw\") pod \"b9095fd0-5447-457c-995f-490ae08411fb\" (UID: \"b9095fd0-5447-457c-995f-490ae08411fb\") "
Sep 11 13:14:05.970796 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.970779    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:05.970865 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.970806    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jh9m8\" (UniqueName: \"kubernetes.io/projected/ba879cbb-9f17-4ad7-bef0-e5b03c903627-kube-api-access-jh9m8\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:05.970865 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.970822    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba879cbb-9f17-4ad7-bef0-e5b03c903627-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:05.970865 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.970834    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba879cbb-9f17-4ad7-bef0-e5b03c903627-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:05.970865 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.970776    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9095fd0-5447-457c-995f-490ae08411fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b9095fd0-5447-457c-995f-490ae08411fb" (UID: "b9095fd0-5447-457c-995f-490ae08411fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:14:05.972497 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:05.972474    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9095fd0-5447-457c-995f-490ae08411fb-kube-api-access-qx5mw" (OuterVolumeSpecName: "kube-api-access-qx5mw") pod "b9095fd0-5447-457c-995f-490ae08411fb" (UID: "b9095fd0-5447-457c-995f-490ae08411fb"). InnerVolumeSpecName "kube-api-access-qx5mw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:14:06.071336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.071241    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9095fd0-5447-457c-995f-490ae08411fb-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:06.071336 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.071277    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx5mw\" (UniqueName: \"kubernetes.io/projected/b9095fd0-5447-457c-995f-490ae08411fb-kube-api-access-qx5mw\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:06.719199 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.719172    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"
Sep 11 13:14:06.719199 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.719184    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f" event={"ID":"b9095fd0-5447-457c-995f-490ae08411fb","Type":"ContainerDied","Data":"fc01c09c7fa95295b74d9bd39681d645fc1d9f77793de556e74dcffaeb61882a"}
Sep 11 13:14:06.719863 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.719257    2564 scope.go:117] "RemoveContainer" containerID="1cb226180bcb38e649006c1c5e23d6e0136613d07b0030160e05404217cba727"
Sep 11 13:14:06.720830 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.720806    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"
Sep 11 13:14:06.720946 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.720806    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb" event={"ID":"ba879cbb-9f17-4ad7-bef0-e5b03c903627","Type":"ContainerDied","Data":"f32586a80bdf180df603e4dd4152541341cf7dd60e5388fb79768e5b94024058"}
Sep 11 13:14:06.729200 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.729177    2564 scope.go:117] "RemoveContainer" containerID="ae447c6aef062c114dfacd4f255262898343d08e6197d377913c78078d64d01c"
Sep 11 13:14:06.755620 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.755531    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"]
Sep 11 13:14:06.764432 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.764395    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-940f81-56b5d6fdbd-6lflb"]
Sep 11 13:14:06.798185 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.798149    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"]
Sep 11 13:14:06.803802 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:06.803767    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-940f81-cf45874bb-lw56f"]
Sep 11 13:14:08.483795 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:08.483760    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9095fd0-5447-457c-995f-490ae08411fb" path="/var/lib/kubelet/pods/b9095fd0-5447-457c-995f-490ae08411fb/volumes"
Sep 11 13:14:08.484190 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:08.484177    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba879cbb-9f17-4ad7-bef0-e5b03c903627" path="/var/lib/kubelet/pods/ba879cbb-9f17-4ad7-bef0-e5b03c903627/volumes"
Sep 11 13:14:29.989817 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:29.989770    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"]
Sep 11 13:14:29.990245 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:29.990077    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t" podUID="f3d97fcb-4261-4ef7-9c21-970b53082f4c" containerName="payload-processing" containerID="cri-o://6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715" gracePeriod=30
Sep 11 13:14:30.003554 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.003520    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"]
Sep 11 13:14:30.003864 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.003831    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck" podUID="fc3049f1-f1d1-446a-b820-06f250ec9616" containerName="payload-pre-processing" containerID="cri-o://7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd" gracePeriod=30
Sep 11 13:14:30.290768 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.290738    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:14:30.293892 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.293866    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:14:30.414264 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.414234    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtjh2\" (UniqueName: \"kubernetes.io/projected/f3d97fcb-4261-4ef7-9c21-970b53082f4c-kube-api-access-gtjh2\") pod \"f3d97fcb-4261-4ef7-9c21-970b53082f4c\" (UID: \"f3d97fcb-4261-4ef7-9c21-970b53082f4c\") "
Sep 11 13:14:30.414450 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.414298    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2wvz\" (UniqueName: \"kubernetes.io/projected/fc3049f1-f1d1-446a-b820-06f250ec9616-kube-api-access-s2wvz\") pod \"fc3049f1-f1d1-446a-b820-06f250ec9616\" (UID: \"fc3049f1-f1d1-446a-b820-06f250ec9616\") "
Sep 11 13:14:30.414450 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.414367    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d97fcb-4261-4ef7-9c21-970b53082f4c-plugins-config-volume\") pod \"f3d97fcb-4261-4ef7-9c21-970b53082f4c\" (UID: \"f3d97fcb-4261-4ef7-9c21-970b53082f4c\") "
Sep 11 13:14:30.414450 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.414393    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc3049f1-f1d1-446a-b820-06f250ec9616-plugins-config-volume\") pod \"fc3049f1-f1d1-446a-b820-06f250ec9616\" (UID: \"fc3049f1-f1d1-446a-b820-06f250ec9616\") "
Sep 11 13:14:30.414824 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.414797    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3d97fcb-4261-4ef7-9c21-970b53082f4c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f3d97fcb-4261-4ef7-9c21-970b53082f4c" (UID: "f3d97fcb-4261-4ef7-9c21-970b53082f4c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:14:30.414934 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.414835    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc3049f1-f1d1-446a-b820-06f250ec9616-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc3049f1-f1d1-446a-b820-06f250ec9616" (UID: "fc3049f1-f1d1-446a-b820-06f250ec9616"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:14:30.416416 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.416396    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc3049f1-f1d1-446a-b820-06f250ec9616-kube-api-access-s2wvz" (OuterVolumeSpecName: "kube-api-access-s2wvz") pod "fc3049f1-f1d1-446a-b820-06f250ec9616" (UID: "fc3049f1-f1d1-446a-b820-06f250ec9616"). InnerVolumeSpecName "kube-api-access-s2wvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:14:30.416495 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.416425    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d97fcb-4261-4ef7-9c21-970b53082f4c-kube-api-access-gtjh2" (OuterVolumeSpecName: "kube-api-access-gtjh2") pod "f3d97fcb-4261-4ef7-9c21-970b53082f4c" (UID: "f3d97fcb-4261-4ef7-9c21-970b53082f4c"). InnerVolumeSpecName "kube-api-access-gtjh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:14:30.515574 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.515484    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d97fcb-4261-4ef7-9c21-970b53082f4c-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:30.515574 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.515515    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc3049f1-f1d1-446a-b820-06f250ec9616-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:30.515574 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.515524    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtjh2\" (UniqueName: \"kubernetes.io/projected/f3d97fcb-4261-4ef7-9c21-970b53082f4c-kube-api-access-gtjh2\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:30.515574 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.515533    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2wvz\" (UniqueName: \"kubernetes.io/projected/fc3049f1-f1d1-446a-b820-06f250ec9616-kube-api-access-s2wvz\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:30.868028 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.867995    2564 generic.go:356] "Generic (PLEG): container finished" podID="fc3049f1-f1d1-446a-b820-06f250ec9616" containerID="7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd" exitCode=0
Sep 11 13:14:30.868226 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.868040    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"
Sep 11 13:14:30.868226 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.868080    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck" event={"ID":"fc3049f1-f1d1-446a-b820-06f250ec9616","Type":"ContainerDied","Data":"7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd"}
Sep 11 13:14:30.868226 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.868138    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck" event={"ID":"fc3049f1-f1d1-446a-b820-06f250ec9616","Type":"ContainerDied","Data":"573c4a339a214fa0bba0dbdc6a04f7de788d82cfccbbd395e450b66bd47cf6dc"}
Sep 11 13:14:30.868226 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.868172    2564 scope.go:117] "RemoveContainer" containerID="7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd"
Sep 11 13:14:30.869530 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.869510    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3d97fcb-4261-4ef7-9c21-970b53082f4c" containerID="6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715" exitCode=0
Sep 11 13:14:30.869631 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.869548    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"
Sep 11 13:14:30.869631 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.869595    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t" event={"ID":"f3d97fcb-4261-4ef7-9c21-970b53082f4c","Type":"ContainerDied","Data":"6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715"}
Sep 11 13:14:30.869806 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.869664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t" event={"ID":"f3d97fcb-4261-4ef7-9c21-970b53082f4c","Type":"ContainerDied","Data":"8bfbcc01c78e010ff35de41fe6ec7063e01d40639293e91b8b802031d214ebea"}
Sep 11 13:14:30.877998 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.877978    2564 scope.go:117] "RemoveContainer" containerID="7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd"
Sep 11 13:14:30.878362 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:14:30.878344    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd\": container with ID starting with 7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd not found: ID does not exist" containerID="7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd"
Sep 11 13:14:30.878442 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.878375    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd"} err="failed to get container status \"7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd\": rpc error: code = NotFound desc = could not find container \"7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd\": container with ID starting with 7294772d27f28d1f8bac2df2a2daf3ec3bb38039ae975df9afd704bc67b00edd not found: ID does not exist"
Sep 11 13:14:30.878442 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.878396    2564 scope.go:117] "RemoveContainer" containerID="6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715"
Sep 11 13:14:30.887628 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.887605    2564 scope.go:117] "RemoveContainer" containerID="6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715"
Sep 11 13:14:30.887979 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:14:30.887959    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715\": container with ID starting with 6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715 not found: ID does not exist" containerID="6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715"
Sep 11 13:14:30.888047 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.887991    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715"} err="failed to get container status \"6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715\": rpc error: code = NotFound desc = could not find container \"6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715\": container with ID starting with 6e714cafe85a457c6cc0f04065f6770bbaac6ae745c10e077759b22d83d94715 not found: ID does not exist"
Sep 11 13:14:30.888518 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.888497    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"]
Sep 11 13:14:30.892692 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.892629    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ccae06-557bcb45f7-dbf7t"]
Sep 11 13:14:30.902773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.902736    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"]
Sep 11 13:14:30.906869 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:30.906838    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ccae06-678c4647d-vg2ck"]
Sep 11 13:14:32.485115 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:32.485083    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d97fcb-4261-4ef7-9c21-970b53082f4c" path="/var/lib/kubelet/pods/f3d97fcb-4261-4ef7-9c21-970b53082f4c/volumes"
Sep 11 13:14:32.485497 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:32.485449    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc3049f1-f1d1-446a-b820-06f250ec9616" path="/var/lib/kubelet/pods/fc3049f1-f1d1-446a-b820-06f250ec9616/volumes"
Sep 11 13:14:47.597407 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.597367    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"]
Sep 11 13:14:47.598043 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.597623    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr" podUID="059ca660-b6e8-4c7d-b429-530e37f6c863" containerName="maas-api" containerID="cri-o://624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd" gracePeriod=30
Sep 11 13:14:47.675293 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.675249    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"]
Sep 11 13:14:47.675578 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.675522    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8" podUID="653d8a01-2f0e-4fdf-9d7b-cae0f796c4af" containerName="payload-processing" containerID="cri-o://f0b53e5034f3fb2430d7b2f4a91844caebdbd65f2678a9a90a15a3de830a0932" gracePeriod=30
Sep 11 13:14:47.957413 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.957385    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:14:47.979357 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.979296    2564 generic.go:356] "Generic (PLEG): container finished" podID="059ca660-b6e8-4c7d-b429-530e37f6c863" containerID="624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd" exitCode=0
Sep 11 13:14:47.979357 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.979341    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"
Sep 11 13:14:47.979782 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.979495    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr" event={"ID":"059ca660-b6e8-4c7d-b429-530e37f6c863","Type":"ContainerDied","Data":"624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd"}
Sep 11 13:14:47.979782 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.979545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr" event={"ID":"059ca660-b6e8-4c7d-b429-530e37f6c863","Type":"ContainerDied","Data":"a0e6ff35992c852e3ebf213ea6578d2bf2466eb13ba20cdd2cd5b343fe28ca00"}
Sep 11 13:14:47.979782 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.979694    2564 scope.go:117] "RemoveContainer" containerID="624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd"
Sep 11 13:14:47.981486 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.981414    2564 generic.go:356] "Generic (PLEG): container finished" podID="653d8a01-2f0e-4fdf-9d7b-cae0f796c4af" containerID="f0b53e5034f3fb2430d7b2f4a91844caebdbd65f2678a9a90a15a3de830a0932" exitCode=0
Sep 11 13:14:47.981813 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.981761    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8" event={"ID":"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af","Type":"ContainerDied","Data":"f0b53e5034f3fb2430d7b2f4a91844caebdbd65f2678a9a90a15a3de830a0932"}
Sep 11 13:14:47.989774 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.989694    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-maas-api-tls\") pod \"059ca660-b6e8-4c7d-b429-530e37f6c863\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") "
Sep 11 13:14:47.990152 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.989942    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-metrics-certs\") pod \"059ca660-b6e8-4c7d-b429-530e37f6c863\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") "
Sep 11 13:14:47.990152 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.990077    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/059ca660-b6e8-4c7d-b429-530e37f6c863-openshift-service-ca\") pod \"059ca660-b6e8-4c7d-b429-530e37f6c863\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") "
Sep 11 13:14:47.990152 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.990115    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q2vq\" (UniqueName: \"kubernetes.io/projected/059ca660-b6e8-4c7d-b429-530e37f6c863-kube-api-access-7q2vq\") pod \"059ca660-b6e8-4c7d-b429-530e37f6c863\" (UID: \"059ca660-b6e8-4c7d-b429-530e37f6c863\") "
Sep 11 13:14:47.990858 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.990800    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/059ca660-b6e8-4c7d-b429-530e37f6c863-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "059ca660-b6e8-4c7d-b429-530e37f6c863" (UID: "059ca660-b6e8-4c7d-b429-530e37f6c863"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:14:47.992497 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.992472    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "059ca660-b6e8-4c7d-b429-530e37f6c863" (UID: "059ca660-b6e8-4c7d-b429-530e37f6c863"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:14:47.992991 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.992963    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/059ca660-b6e8-4c7d-b429-530e37f6c863-kube-api-access-7q2vq" (OuterVolumeSpecName: "kube-api-access-7q2vq") pod "059ca660-b6e8-4c7d-b429-530e37f6c863" (UID: "059ca660-b6e8-4c7d-b429-530e37f6c863"). InnerVolumeSpecName "kube-api-access-7q2vq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:14:47.993089 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:47.993045    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "059ca660-b6e8-4c7d-b429-530e37f6c863" (UID: "059ca660-b6e8-4c7d-b429-530e37f6c863"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:14:48.001922 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.001901    2564 scope.go:117] "RemoveContainer" containerID="624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd"
Sep 11 13:14:48.002209 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:14:48.002188    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd\": container with ID starting with 624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd not found: ID does not exist" containerID="624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd"
Sep 11 13:14:48.002316 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.002217    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd"} err="failed to get container status \"624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd\": rpc error: code = NotFound desc = could not find container \"624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd\": container with ID starting with 624477f63144f7dadd1925504278cbd83beb619d0125f9ad55ad27516d4093dd not found: ID does not exist"
Sep 11 13:14:48.005222 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.005197    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:14:48.090992 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.090953    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-plugins-config-volume\") pod \"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af\" (UID: \"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af\") "
Sep 11 13:14:48.091184 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.091040    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wsw6\" (UniqueName: \"kubernetes.io/projected/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-kube-api-access-9wsw6\") pod \"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af\" (UID: \"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af\") "
Sep 11 13:14:48.091253 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.091214    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/059ca660-b6e8-4c7d-b429-530e37f6c863-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:48.091253 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.091224    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q2vq\" (UniqueName: \"kubernetes.io/projected/059ca660-b6e8-4c7d-b429-530e37f6c863-kube-api-access-7q2vq\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:48.091253 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.091233    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:48.091253 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.091242    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059ca660-b6e8-4c7d-b429-530e37f6c863-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:48.091470 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.091437    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "653d8a01-2f0e-4fdf-9d7b-cae0f796c4af" (UID: "653d8a01-2f0e-4fdf-9d7b-cae0f796c4af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:14:48.093701 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.093658    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-kube-api-access-9wsw6" (OuterVolumeSpecName: "kube-api-access-9wsw6") pod "653d8a01-2f0e-4fdf-9d7b-cae0f796c4af" (UID: "653d8a01-2f0e-4fdf-9d7b-cae0f796c4af"). InnerVolumeSpecName "kube-api-access-9wsw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:14:48.192493 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.192393    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:48.192493 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.192426    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wsw6\" (UniqueName: \"kubernetes.io/projected/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af-kube-api-access-9wsw6\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:14:48.307481 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.307445    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"]
Sep 11 13:14:48.313097 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.313064    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b81d9e-676f78d9bb-b2rmr"]
Sep 11 13:14:48.483776 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.483694    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="059ca660-b6e8-4c7d-b429-530e37f6c863" path="/var/lib/kubelet/pods/059ca660-b6e8-4c7d-b429-530e37f6c863/volumes"
Sep 11 13:14:48.988203 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.988169    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"
Sep 11 13:14:48.988203 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.988184    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8" event={"ID":"653d8a01-2f0e-4fdf-9d7b-cae0f796c4af","Type":"ContainerDied","Data":"20ebc951c779f4010aa048baf48613fcfe325aefef8b42450591b09336e7f2b9"}
Sep 11 13:14:48.988803 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:48.988257    2564 scope.go:117] "RemoveContainer" containerID="f0b53e5034f3fb2430d7b2f4a91844caebdbd65f2678a9a90a15a3de830a0932"
Sep 11 13:14:49.028606 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:49.028568    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"]
Sep 11 13:14:49.034215 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:49.034177    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b81d9e-776b687567-dg7x8"]
Sep 11 13:14:50.483878 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:14:50.483844    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="653d8a01-2f0e-4fdf-9d7b-cae0f796c4af" path="/var/lib/kubelet/pods/653d8a01-2f0e-4fdf-9d7b-cae0f796c4af/volumes"
Sep 11 13:15:06.440266 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.440225    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"]
Sep 11 13:15:06.440799 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.440511    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2" podUID="24c88e64-5ea8-4f2d-93b5-56ede95aaa83" containerName="maas-api" containerID="cri-o://403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4" gracePeriod=30
Sep 11 13:15:06.724866 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.724838    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:15:06.875005 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.874972    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-maas-api-tls\") pod \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") "
Sep 11 13:15:06.875194 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.875081    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-metrics-certs\") pod \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") "
Sep 11 13:15:06.875194 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.875154    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-openshift-service-ca\") pod \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") "
Sep 11 13:15:06.875338 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.875318    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbhvz\" (UniqueName: \"kubernetes.io/projected/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-kube-api-access-xbhvz\") pod \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\" (UID: \"24c88e64-5ea8-4f2d-93b5-56ede95aaa83\") "
Sep 11 13:15:06.875469 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.875442    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "24c88e64-5ea8-4f2d-93b5-56ede95aaa83" (UID: "24c88e64-5ea8-4f2d-93b5-56ede95aaa83"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:15:06.875721 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.875698    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:15:06.877366 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.877342    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "24c88e64-5ea8-4f2d-93b5-56ede95aaa83" (UID: "24c88e64-5ea8-4f2d-93b5-56ede95aaa83"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:15:06.877709 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.877667    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "24c88e64-5ea8-4f2d-93b5-56ede95aaa83" (UID: "24c88e64-5ea8-4f2d-93b5-56ede95aaa83"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:15:06.877867 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.877841    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-kube-api-access-xbhvz" (OuterVolumeSpecName: "kube-api-access-xbhvz") pod "24c88e64-5ea8-4f2d-93b5-56ede95aaa83" (UID: "24c88e64-5ea8-4f2d-93b5-56ede95aaa83"). InnerVolumeSpecName "kube-api-access-xbhvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:15:06.977013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.976932    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:15:06.977013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.976959    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:15:06.977013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:06.976968    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbhvz\" (UniqueName: \"kubernetes.io/projected/24c88e64-5ea8-4f2d-93b5-56ede95aaa83-kube-api-access-xbhvz\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:15:07.100110 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:07.100069    2564 generic.go:356] "Generic (PLEG): container finished" podID="24c88e64-5ea8-4f2d-93b5-56ede95aaa83" containerID="403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4" exitCode=0
Sep 11 13:15:07.100110 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:07.100112    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"
Sep 11 13:15:07.100371 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:07.100148    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2" event={"ID":"24c88e64-5ea8-4f2d-93b5-56ede95aaa83","Type":"ContainerDied","Data":"403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4"}
Sep 11 13:15:07.100371 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:07.100197    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2" event={"ID":"24c88e64-5ea8-4f2d-93b5-56ede95aaa83","Type":"ContainerDied","Data":"788482868dbf7a4e096126b0561b2b8bfa75804eb3d2d220a210d5ad995fedc8"}
Sep 11 13:15:07.100371 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:07.100219    2564 scope.go:117] "RemoveContainer" containerID="403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4"
Sep 11 13:15:07.110292 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:07.110272    2564 scope.go:117] "RemoveContainer" containerID="403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4"
Sep 11 13:15:07.110634 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:15:07.110614    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4\": container with ID starting with 403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4 not found: ID does not exist" containerID="403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4"
Sep 11 13:15:07.110756 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:07.110674    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4"} err="failed to get container status \"403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4\": rpc error: code = NotFound desc = could not find container \"403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4\": container with ID starting with 403e98912feae2756ce878e99c9878f480134d42d6080a298ba8eaeab79c11e4 not found: ID does not exist"
Sep 11 13:15:07.135634 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:07.135600    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"]
Sep 11 13:15:07.139657 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:07.139610    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c542a-c45f95cd8-pklb2"]
Sep 11 13:15:08.483940 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:15:08.483897    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24c88e64-5ea8-4f2d-93b5-56ede95aaa83" path="/var/lib/kubelet/pods/24c88e64-5ea8-4f2d-93b5-56ede95aaa83/volumes"
Sep 11 13:16:15.138016 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.137935    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"]
Sep 11 13:16:15.138707 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138594    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d97fcb-4261-4ef7-9c21-970b53082f4c" containerName="payload-processing"
Sep 11 13:16:15.138707 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138615    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d97fcb-4261-4ef7-9c21-970b53082f4c" containerName="payload-processing"
Sep 11 13:16:15.138707 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138673    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba879cbb-9f17-4ad7-bef0-e5b03c903627" containerName="maas-api"
Sep 11 13:16:15.138707 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138681    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba879cbb-9f17-4ad7-bef0-e5b03c903627" containerName="maas-api"
Sep 11 13:16:15.138707 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138691    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc3049f1-f1d1-446a-b820-06f250ec9616" containerName="payload-pre-processing"
Sep 11 13:16:15.138707 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138697    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc3049f1-f1d1-446a-b820-06f250ec9616" containerName="payload-pre-processing"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138714    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c88e64-5ea8-4f2d-93b5-56ede95aaa83" containerName="maas-api"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138720    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c88e64-5ea8-4f2d-93b5-56ede95aaa83" containerName="maas-api"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138728    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9095fd0-5447-457c-995f-490ae08411fb" containerName="payload-pre-processing"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138736    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9095fd0-5447-457c-995f-490ae08411fb" containerName="payload-pre-processing"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138758    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="653d8a01-2f0e-4fdf-9d7b-cae0f796c4af" containerName="payload-processing"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138763    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="653d8a01-2f0e-4fdf-9d7b-cae0f796c4af" containerName="payload-processing"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138772    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="059ca660-b6e8-4c7d-b429-530e37f6c863" containerName="maas-api"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138777    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="059ca660-b6e8-4c7d-b429-530e37f6c863" containerName="maas-api"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138848    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c88e64-5ea8-4f2d-93b5-56ede95aaa83" containerName="maas-api"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138857    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba879cbb-9f17-4ad7-bef0-e5b03c903627" containerName="maas-api"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138868    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="653d8a01-2f0e-4fdf-9d7b-cae0f796c4af" containerName="payload-processing"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138875    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="059ca660-b6e8-4c7d-b429-530e37f6c863" containerName="maas-api"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138880    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9095fd0-5447-457c-995f-490ae08411fb" containerName="payload-pre-processing"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138890    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc3049f1-f1d1-446a-b820-06f250ec9616" containerName="payload-pre-processing"
Sep 11 13:16:15.139130 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.138899    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d97fcb-4261-4ef7-9c21-970b53082f4c" containerName="payload-processing"
Sep 11 13:16:15.142316 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.142289    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.145165 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.145138    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-37c1cf-serving-cert\""
Sep 11 13:16:15.159809 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.159771    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"]
Sep 11 13:16:15.220309 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.220258    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llbnr\" (UniqueName: \"kubernetes.io/projected/4b763dfe-4ac4-4253-9696-6e7b4c218efa-kube-api-access-llbnr\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.220492 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.220391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-metrics-certs\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.220492 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.220459    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-maas-api-tls\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.220591 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.220531    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b763dfe-4ac4-4253-9696-6e7b4c218efa-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.321594 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.321558    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-maas-api-tls\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.321832 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.321611    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b763dfe-4ac4-4253-9696-6e7b4c218efa-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.321832 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.321679    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llbnr\" (UniqueName: \"kubernetes.io/projected/4b763dfe-4ac4-4253-9696-6e7b4c218efa-kube-api-access-llbnr\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.321832 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.321769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-metrics-certs\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.322246 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.322223    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b763dfe-4ac4-4253-9696-6e7b4c218efa-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.324245 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.324219    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-metrics-certs\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.324390 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.324370    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-maas-api-tls\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.345337 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.345305    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llbnr\" (UniqueName: \"kubernetes.io/projected/4b763dfe-4ac4-4253-9696-6e7b4c218efa-kube-api-access-llbnr\") pod \"maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.455631 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.455534    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:15.777622 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:15.777591    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"]
Sep 11 13:16:15.779805 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:16:15.779772    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b763dfe_4ac4_4253_9696_6e7b4c218efa.slice/crio-a5abc6d9729dd4a57632c92910820332513743de05daab8471511a297d935e82 WatchSource:0}: Error finding container a5abc6d9729dd4a57632c92910820332513743de05daab8471511a297d935e82: Status 404 returned error can't find the container with id a5abc6d9729dd4a57632c92910820332513743de05daab8471511a297d935e82
Sep 11 13:16:16.491611 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.491519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg" event={"ID":"4b763dfe-4ac4-4253-9696-6e7b4c218efa","Type":"ContainerStarted","Data":"dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a"}
Sep 11 13:16:16.492017 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.491673    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:16.492017 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.491721    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg" event={"ID":"4b763dfe-4ac4-4253-9696-6e7b4c218efa","Type":"ContainerStarted","Data":"a5abc6d9729dd4a57632c92910820332513743de05daab8471511a297d935e82"}
Sep 11 13:16:16.561857 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.561803    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg" podStartSLOduration=1.149905378 podStartE2EDuration="1.561783478s" totalImagesPullingTime="411.8781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:16:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:16:16.560491734 +0000 UTC m=+1744.622798684" watchObservedRunningTime="2026-09-11 13:16:16.561783478 +0000 UTC m=+1744.624090474"
Sep 11 13:16:16.605332 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.605291    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"]
Sep 11 13:16:16.609268 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.609246    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:16:16.612415 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.612389    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-37c1cf\""
Sep 11 13:16:16.613853 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.613826    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-37c1cf-dockercfg-qkpwg\""
Sep 11 13:16:16.637024 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.636982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slm4v\" (UniqueName: \"kubernetes.io/projected/d975aa47-763c-4684-ac05-7b35195473a3-kube-api-access-slm4v\") pod \"payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6\" (UID: \"d975aa47-763c-4684-ac05-7b35195473a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:16:16.637212 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.637103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d975aa47-763c-4684-ac05-7b35195473a3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6\" (UID: \"d975aa47-763c-4684-ac05-7b35195473a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:16:16.655237 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.655161    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"]
Sep 11 13:16:16.738101 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.738066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d975aa47-763c-4684-ac05-7b35195473a3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6\" (UID: \"d975aa47-763c-4684-ac05-7b35195473a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:16:16.738259 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.738233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slm4v\" (UniqueName: \"kubernetes.io/projected/d975aa47-763c-4684-ac05-7b35195473a3-kube-api-access-slm4v\") pod \"payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6\" (UID: \"d975aa47-763c-4684-ac05-7b35195473a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:16:16.738797 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.738778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d975aa47-763c-4684-ac05-7b35195473a3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6\" (UID: \"d975aa47-763c-4684-ac05-7b35195473a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:16:16.763752 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.763671    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slm4v\" (UniqueName: \"kubernetes.io/projected/d975aa47-763c-4684-ac05-7b35195473a3-kube-api-access-slm4v\") pod \"payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6\" (UID: \"d975aa47-763c-4684-ac05-7b35195473a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:16:16.921743 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:16.921696    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:16:17.212232 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:17.212204    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"]
Sep 11 13:16:17.213592 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:16:17.213558    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd975aa47_763c_4684_ac05_7b35195473a3.slice/crio-d5ceeb1bc2287f517dcc4ecf97c41580d1a412372368ffb9a06e0156e8f79c89 WatchSource:0}: Error finding container d5ceeb1bc2287f517dcc4ecf97c41580d1a412372368ffb9a06e0156e8f79c89: Status 404 returned error can't find the container with id d5ceeb1bc2287f517dcc4ecf97c41580d1a412372368ffb9a06e0156e8f79c89
Sep 11 13:16:17.499229 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:17.499128    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6" event={"ID":"d975aa47-763c-4684-ac05-7b35195473a3","Type":"ContainerStarted","Data":"dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323"}
Sep 11 13:16:17.499229 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:17.499191    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6" event={"ID":"d975aa47-763c-4684-ac05-7b35195473a3","Type":"ContainerStarted","Data":"d5ceeb1bc2287f517dcc4ecf97c41580d1a412372368ffb9a06e0156e8f79c89"}
Sep 11 13:16:17.499775 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:17.499506    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:16:17.542578 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:17.542515    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6" podStartSLOduration=1.542488995 podStartE2EDuration="1.542488995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:16:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:16:17.541042071 +0000 UTC m=+1745.603349052" watchObservedRunningTime="2026-09-11 13:16:17.542488995 +0000 UTC m=+1745.604795956"
Sep 11 13:16:22.505161 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:22.505132    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:16:28.506736 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:28.506702    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:16:49.103056 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.103018    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"]
Sep 11 13:16:49.112819 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.112786    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.121241 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.121213    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"]
Sep 11 13:16:49.124125 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.124103    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-06286a-serving-cert\""
Sep 11 13:16:49.263372 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.263328    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l2k5\" (UniqueName: \"kubernetes.io/projected/e16f4e99-6159-42d5-acfb-ce99e52c69cd-kube-api-access-2l2k5\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.263582 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.263397    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e16f4e99-6159-42d5-acfb-ce99e52c69cd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.263582 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.263435    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-metrics-certs\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.263582 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.263476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.364315 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.364228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l2k5\" (UniqueName: \"kubernetes.io/projected/e16f4e99-6159-42d5-acfb-ce99e52c69cd-kube-api-access-2l2k5\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.364315 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.364278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e16f4e99-6159-42d5-acfb-ce99e52c69cd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.364315 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.364301    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-metrics-certs\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.364553 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.364329    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.364553 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:16:49.364428    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-serving-cert: secret "maas-api-e2e-ipp-b-06286a-serving-cert" not found
Sep 11 13:16:49.364553 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:16:49.364494    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-maas-api-tls podName:e16f4e99-6159-42d5-acfb-ce99e52c69cd nodeName:}" failed. No retries permitted until 2026-09-11 13:16:49.864475694 +0000 UTC m=+1777.926782631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-maas-api-tls") pod "maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l" (UID: "e16f4e99-6159-42d5-acfb-ce99e52c69cd") : secret "maas-api-e2e-ipp-b-06286a-serving-cert" not found
Sep 11 13:16:49.364977 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.364954    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e16f4e99-6159-42d5-acfb-ce99e52c69cd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.366855 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.366835    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-metrics-certs\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.375406 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.375383    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l2k5\" (UniqueName: \"kubernetes.io/projected/e16f4e99-6159-42d5-acfb-ce99e52c69cd-kube-api-access-2l2k5\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.870102 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.870063    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:49.872461 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:49.872434    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:50.026864 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:50.026826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:50.308195 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:50.308112    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"]
Sep 11 13:16:50.311135 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:16:50.311104    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16f4e99_6159_42d5_acfb_ce99e52c69cd.slice/crio-61df95fee010f76e96a14064d7521159151e93756e54890567b63d7ab010be99 WatchSource:0}: Error finding container 61df95fee010f76e96a14064d7521159151e93756e54890567b63d7ab010be99: Status 404 returned error can't find the container with id 61df95fee010f76e96a14064d7521159151e93756e54890567b63d7ab010be99
Sep 11 13:16:50.686250 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:50.686222    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l" event={"ID":"e16f4e99-6159-42d5-acfb-ce99e52c69cd","Type":"ContainerStarted","Data":"61df95fee010f76e96a14064d7521159151e93756e54890567b63d7ab010be99"}
Sep 11 13:16:51.694340 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:51.694306    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l" event={"ID":"e16f4e99-6159-42d5-acfb-ce99e52c69cd","Type":"ContainerStarted","Data":"2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e"}
Sep 11 13:16:51.694763 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:51.694429    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:16:51.715453 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:51.715395    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l" podStartSLOduration=2.346468794 podStartE2EDuration="2.715375739s" totalImagesPullingTime="368.906945ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:16:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:16:51.713063661 +0000 UTC m=+1779.775370620" watchObservedRunningTime="2026-09-11 13:16:51.715375739 +0000 UTC m=+1779.777682697"
Sep 11 13:16:57.706047 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:16:57.706017    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:17:12.581467 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:17:12.581432    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:17:12.587400 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:17:12.587371    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:17:18.117931 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:17:18.117897    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dsj2b_82825f9e-f8bd-4fcc-b301-eaa5031c3f6e/payload-processing/0.log"
Sep 11 13:18:01.339777 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:01.339747    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6_d975aa47-763c-4684-ac05-7b35195473a3/payload-processing/0.log"
Sep 11 13:18:01.472659 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:01.472616    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dsj2b_82825f9e-f8bd-4fcc-b301-eaa5031c3f6e/payload-processing/0.log"
Sep 11 13:18:07.274795 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.274753    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"]
Sep 11 13:18:07.278680 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.278629    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:07.281330 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.281302    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-338b23\""
Sep 11 13:18:07.281680 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.281630    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-338b23-dockercfg-9c4lw\""
Sep 11 13:18:07.291387 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.291358    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"]
Sep 11 13:18:07.338347 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.338311    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"]
Sep 11 13:18:07.341993 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.341963    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:07.356867 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.356828    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"]
Sep 11 13:18:07.426325 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.426292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64xdh\" (UniqueName: \"kubernetes.io/projected/a2f16b97-a816-480c-adce-d097778befac-kube-api-access-64xdh\") pod \"payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw\" (UID: \"a2f16b97-a816-480c-adce-d097778befac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:07.426507 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.426340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2f16b97-a816-480c-adce-d097778befac-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw\" (UID: \"a2f16b97-a816-480c-adce-d097778befac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:07.527208 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.527119    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2f16b97-a816-480c-adce-d097778befac-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw\" (UID: \"a2f16b97-a816-480c-adce-d097778befac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:07.527208 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.527175    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcjgv\" (UniqueName: \"kubernetes.io/projected/da2891fc-7b28-4e71-a27a-9258cde2e185-kube-api-access-kcjgv\") pod \"payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k\" (UID: \"da2891fc-7b28-4e71-a27a-9258cde2e185\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:07.527399 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.527322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64xdh\" (UniqueName: \"kubernetes.io/projected/a2f16b97-a816-480c-adce-d097778befac-kube-api-access-64xdh\") pod \"payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw\" (UID: \"a2f16b97-a816-480c-adce-d097778befac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:07.527399 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.527349    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da2891fc-7b28-4e71-a27a-9258cde2e185-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k\" (UID: \"da2891fc-7b28-4e71-a27a-9258cde2e185\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:07.527921 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.527900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2f16b97-a816-480c-adce-d097778befac-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw\" (UID: \"a2f16b97-a816-480c-adce-d097778befac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:07.536498 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.536461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64xdh\" (UniqueName: \"kubernetes.io/projected/a2f16b97-a816-480c-adce-d097778befac-kube-api-access-64xdh\") pod \"payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw\" (UID: \"a2f16b97-a816-480c-adce-d097778befac\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:07.590054 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.590019    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:07.628010 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.627974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcjgv\" (UniqueName: \"kubernetes.io/projected/da2891fc-7b28-4e71-a27a-9258cde2e185-kube-api-access-kcjgv\") pod \"payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k\" (UID: \"da2891fc-7b28-4e71-a27a-9258cde2e185\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:07.628238 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.628116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da2891fc-7b28-4e71-a27a-9258cde2e185-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k\" (UID: \"da2891fc-7b28-4e71-a27a-9258cde2e185\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:07.628891 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.628865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da2891fc-7b28-4e71-a27a-9258cde2e185-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k\" (UID: \"da2891fc-7b28-4e71-a27a-9258cde2e185\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:07.637228 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.637196    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcjgv\" (UniqueName: \"kubernetes.io/projected/da2891fc-7b28-4e71-a27a-9258cde2e185-kube-api-access-kcjgv\") pod \"payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k\" (UID: \"da2891fc-7b28-4e71-a27a-9258cde2e185\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:07.652891 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.652608    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:07.876831 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.876803    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"]
Sep 11 13:18:07.877696 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:18:07.877661    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f16b97_a816_480c_adce_d097778befac.slice/crio-634ea2535147cebd46d0607d18198c3a39e49a762a265aa7f4d3c093bb72b0b8 WatchSource:0}: Error finding container 634ea2535147cebd46d0607d18198c3a39e49a762a265aa7f4d3c093bb72b0b8: Status 404 returned error can't find the container with id 634ea2535147cebd46d0607d18198c3a39e49a762a265aa7f4d3c093bb72b0b8
Sep 11 13:18:07.950324 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:07.950293    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"]
Sep 11 13:18:07.954828 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:18:07.954801    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda2891fc_7b28_4e71_a27a_9258cde2e185.slice/crio-0d24d49e9bd588325970d938c55468fabc9da10e1a29c9277604f4f2ffe3c2bd WatchSource:0}: Error finding container 0d24d49e9bd588325970d938c55468fabc9da10e1a29c9277604f4f2ffe3c2bd: Status 404 returned error can't find the container with id 0d24d49e9bd588325970d938c55468fabc9da10e1a29c9277604f4f2ffe3c2bd
Sep 11 13:18:08.107427 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:08.107388    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k" event={"ID":"da2891fc-7b28-4e71-a27a-9258cde2e185","Type":"ContainerStarted","Data":"0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014"}
Sep 11 13:18:08.107593 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:08.107453    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k" event={"ID":"da2891fc-7b28-4e71-a27a-9258cde2e185","Type":"ContainerStarted","Data":"0d24d49e9bd588325970d938c55468fabc9da10e1a29c9277604f4f2ffe3c2bd"}
Sep 11 13:18:08.107593 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:08.107482    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:08.109182 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:08.109156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw" event={"ID":"a2f16b97-a816-480c-adce-d097778befac","Type":"ContainerStarted","Data":"a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0"}
Sep 11 13:18:08.109314 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:08.109202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw" event={"ID":"a2f16b97-a816-480c-adce-d097778befac","Type":"ContainerStarted","Data":"634ea2535147cebd46d0607d18198c3a39e49a762a265aa7f4d3c093bb72b0b8"}
Sep 11 13:18:08.109314 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:08.109228    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:08.129996 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:08.129885    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k" podStartSLOduration=1.129862651 podStartE2EDuration="1.129862651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:18:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:18:08.128506653 +0000 UTC m=+1856.190813613" watchObservedRunningTime="2026-09-11 13:18:08.129862651 +0000 UTC m=+1856.192169612"
Sep 11 13:18:08.150773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:08.150716    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw" podStartSLOduration=1.150696169 podStartE2EDuration="1.150696169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:18:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:18:08.15021397 +0000 UTC m=+1856.212520929" watchObservedRunningTime="2026-09-11 13:18:08.150696169 +0000 UTC m=+1856.213003126"
Sep 11 13:18:19.126072 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:19.126038    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:19.126519 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:19.126221    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:37.224095 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.224002    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"]
Sep 11 13:18:37.224714 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.224281    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw" podUID="a2f16b97-a816-480c-adce-d097778befac" containerName="payload-processing" containerID="cri-o://a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0" gracePeriod=30
Sep 11 13:18:37.234270 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.234237    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"]
Sep 11 13:18:37.234588 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.234550    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k" podUID="da2891fc-7b28-4e71-a27a-9258cde2e185" containerName="payload-pre-processing" containerID="cri-o://0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014" gracePeriod=30
Sep 11 13:18:37.513112 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.513083    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:37.550196 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.550156    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:37.622159 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.622124    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2f16b97-a816-480c-adce-d097778befac-plugins-config-volume\") pod \"a2f16b97-a816-480c-adce-d097778befac\" (UID: \"a2f16b97-a816-480c-adce-d097778befac\") "
Sep 11 13:18:37.622341 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.622187    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64xdh\" (UniqueName: \"kubernetes.io/projected/a2f16b97-a816-480c-adce-d097778befac-kube-api-access-64xdh\") pod \"a2f16b97-a816-480c-adce-d097778befac\" (UID: \"a2f16b97-a816-480c-adce-d097778befac\") "
Sep 11 13:18:37.622574 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.622546    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f16b97-a816-480c-adce-d097778befac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a2f16b97-a816-480c-adce-d097778befac" (UID: "a2f16b97-a816-480c-adce-d097778befac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:18:37.624281 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.624258    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2f16b97-a816-480c-adce-d097778befac-kube-api-access-64xdh" (OuterVolumeSpecName: "kube-api-access-64xdh") pod "a2f16b97-a816-480c-adce-d097778befac" (UID: "a2f16b97-a816-480c-adce-d097778befac"). InnerVolumeSpecName "kube-api-access-64xdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:18:37.723021 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.722986    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da2891fc-7b28-4e71-a27a-9258cde2e185-plugins-config-volume\") pod \"da2891fc-7b28-4e71-a27a-9258cde2e185\" (UID: \"da2891fc-7b28-4e71-a27a-9258cde2e185\") "
Sep 11 13:18:37.723209 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.723053    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcjgv\" (UniqueName: \"kubernetes.io/projected/da2891fc-7b28-4e71-a27a-9258cde2e185-kube-api-access-kcjgv\") pod \"da2891fc-7b28-4e71-a27a-9258cde2e185\" (UID: \"da2891fc-7b28-4e71-a27a-9258cde2e185\") "
Sep 11 13:18:37.723325 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.723307    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2f16b97-a816-480c-adce-d097778befac-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:18:37.723370 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.723331    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64xdh\" (UniqueName: \"kubernetes.io/projected/a2f16b97-a816-480c-adce-d097778befac-kube-api-access-64xdh\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:18:37.723370 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.723357    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da2891fc-7b28-4e71-a27a-9258cde2e185-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "da2891fc-7b28-4e71-a27a-9258cde2e185" (UID: "da2891fc-7b28-4e71-a27a-9258cde2e185"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:18:37.725197 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.725179    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da2891fc-7b28-4e71-a27a-9258cde2e185-kube-api-access-kcjgv" (OuterVolumeSpecName: "kube-api-access-kcjgv") pod "da2891fc-7b28-4e71-a27a-9258cde2e185" (UID: "da2891fc-7b28-4e71-a27a-9258cde2e185"). InnerVolumeSpecName "kube-api-access-kcjgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:18:37.824877 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.824784    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da2891fc-7b28-4e71-a27a-9258cde2e185-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:18:37.824877 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:37.824816    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcjgv\" (UniqueName: \"kubernetes.io/projected/da2891fc-7b28-4e71-a27a-9258cde2e185-kube-api-access-kcjgv\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:18:38.293324 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.293286    2564 generic.go:356] "Generic (PLEG): container finished" podID="da2891fc-7b28-4e71-a27a-9258cde2e185" containerID="0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014" exitCode=0
Sep 11 13:18:38.293804 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.293341    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"
Sep 11 13:18:38.293804 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.293378    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k" event={"ID":"da2891fc-7b28-4e71-a27a-9258cde2e185","Type":"ContainerDied","Data":"0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014"}
Sep 11 13:18:38.293804 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.293436    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k" event={"ID":"da2891fc-7b28-4e71-a27a-9258cde2e185","Type":"ContainerDied","Data":"0d24d49e9bd588325970d938c55468fabc9da10e1a29c9277604f4f2ffe3c2bd"}
Sep 11 13:18:38.293804 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.293468    2564 scope.go:117] "RemoveContainer" containerID="0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014"
Sep 11 13:18:38.294906 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.294855    2564 generic.go:356] "Generic (PLEG): container finished" podID="a2f16b97-a816-480c-adce-d097778befac" containerID="a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0" exitCode=0
Sep 11 13:18:38.294971 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.294906    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"
Sep 11 13:18:38.294971 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.294909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw" event={"ID":"a2f16b97-a816-480c-adce-d097778befac","Type":"ContainerDied","Data":"a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0"}
Sep 11 13:18:38.294971 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.294963    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw" event={"ID":"a2f16b97-a816-480c-adce-d097778befac","Type":"ContainerDied","Data":"634ea2535147cebd46d0607d18198c3a39e49a762a265aa7f4d3c093bb72b0b8"}
Sep 11 13:18:38.303017 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.302997    2564 scope.go:117] "RemoveContainer" containerID="0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014"
Sep 11 13:18:38.303347 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:18:38.303321    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014\": container with ID starting with 0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014 not found: ID does not exist" containerID="0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014"
Sep 11 13:18:38.303461 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.303354    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014"} err="failed to get container status \"0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014\": rpc error: code = NotFound desc = could not find container \"0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014\": container with ID starting with 0bb9cd0f2dc456f45822e0b6273869c5888f8aa9049afa2d2adc7f61098f6014 not found: ID does not exist"
Sep 11 13:18:38.303461 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.303371    2564 scope.go:117] "RemoveContainer" containerID="a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0"
Sep 11 13:18:38.312368 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.312346    2564 scope.go:117] "RemoveContainer" containerID="a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0"
Sep 11 13:18:38.312696 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:18:38.312678    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0\": container with ID starting with a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0 not found: ID does not exist" containerID="a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0"
Sep 11 13:18:38.312742 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.312705    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0"} err="failed to get container status \"a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0\": rpc error: code = NotFound desc = could not find container \"a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0\": container with ID starting with a4d98695752cff3a15da09ff36f4e74836c5bea66c0803453d4d6312cf4b74d0 not found: ID does not exist"
Sep 11 13:18:38.320936 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.320904    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"]
Sep 11 13:18:38.324996 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.324969    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-338b23-d84bb68d5-dkbtw"]
Sep 11 13:18:38.336068 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.336034    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"]
Sep 11 13:18:38.341994 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.341962    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-338b23-cc7dd6f58-hg58k"]
Sep 11 13:18:38.485603 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.485569    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2f16b97-a816-480c-adce-d097778befac" path="/var/lib/kubelet/pods/a2f16b97-a816-480c-adce-d097778befac/volumes"
Sep 11 13:18:38.486002 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:18:38.485985    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da2891fc-7b28-4e71-a27a-9258cde2e185" path="/var/lib/kubelet/pods/da2891fc-7b28-4e71-a27a-9258cde2e185/volumes"
Sep 11 13:19:03.862894 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:03.861288    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"]
Sep 11 13:19:03.862894 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:03.862496    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg" podUID="4b763dfe-4ac4-4253-9696-6e7b4c218efa" containerName="maas-api" containerID="cri-o://dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a" gracePeriod=30
Sep 11 13:19:03.939472 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:03.939433    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"]
Sep 11 13:19:03.939770 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:03.939714    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6" podUID="d975aa47-763c-4684-ac05-7b35195473a3" containerName="payload-processing" containerID="cri-o://dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323" gracePeriod=30
Sep 11 13:19:04.209610 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.209581    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:19:04.253520 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.253489    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:19:04.277849 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.277816    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-maas-api-tls\") pod \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") "
Sep 11 13:19:04.278033 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.277906    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-metrics-certs\") pod \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") "
Sep 11 13:19:04.278033 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.277985    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llbnr\" (UniqueName: \"kubernetes.io/projected/4b763dfe-4ac4-4253-9696-6e7b4c218efa-kube-api-access-llbnr\") pod \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") "
Sep 11 13:19:04.278033 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.278010    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b763dfe-4ac4-4253-9696-6e7b4c218efa-openshift-service-ca\") pod \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\" (UID: \"4b763dfe-4ac4-4253-9696-6e7b4c218efa\") "
Sep 11 13:19:04.278621 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.278589    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b763dfe-4ac4-4253-9696-6e7b4c218efa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b763dfe-4ac4-4253-9696-6e7b4c218efa" (UID: "4b763dfe-4ac4-4253-9696-6e7b4c218efa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:19:04.280080 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.280056    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b763dfe-4ac4-4253-9696-6e7b4c218efa" (UID: "4b763dfe-4ac4-4253-9696-6e7b4c218efa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:19:04.280503 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.280482    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4b763dfe-4ac4-4253-9696-6e7b4c218efa" (UID: "4b763dfe-4ac4-4253-9696-6e7b4c218efa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:19:04.280589 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.280523    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b763dfe-4ac4-4253-9696-6e7b4c218efa-kube-api-access-llbnr" (OuterVolumeSpecName: "kube-api-access-llbnr") pod "4b763dfe-4ac4-4253-9696-6e7b4c218efa" (UID: "4b763dfe-4ac4-4253-9696-6e7b4c218efa"). InnerVolumeSpecName "kube-api-access-llbnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:19:04.379099 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.379061    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d975aa47-763c-4684-ac05-7b35195473a3-plugins-config-volume\") pod \"d975aa47-763c-4684-ac05-7b35195473a3\" (UID: \"d975aa47-763c-4684-ac05-7b35195473a3\") "
Sep 11 13:19:04.379278 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.379115    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slm4v\" (UniqueName: \"kubernetes.io/projected/d975aa47-763c-4684-ac05-7b35195473a3-kube-api-access-slm4v\") pod \"d975aa47-763c-4684-ac05-7b35195473a3\" (UID: \"d975aa47-763c-4684-ac05-7b35195473a3\") "
Sep 11 13:19:04.379343 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.379332    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llbnr\" (UniqueName: \"kubernetes.io/projected/4b763dfe-4ac4-4253-9696-6e7b4c218efa-kube-api-access-llbnr\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:19:04.379381 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.379346    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b763dfe-4ac4-4253-9696-6e7b4c218efa-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:19:04.379381 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.379356    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:19:04.379381 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.379365    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b763dfe-4ac4-4253-9696-6e7b4c218efa-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:19:04.379490 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.379434    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d975aa47-763c-4684-ac05-7b35195473a3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d975aa47-763c-4684-ac05-7b35195473a3" (UID: "d975aa47-763c-4684-ac05-7b35195473a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:19:04.381273 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.381251    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d975aa47-763c-4684-ac05-7b35195473a3-kube-api-access-slm4v" (OuterVolumeSpecName: "kube-api-access-slm4v") pod "d975aa47-763c-4684-ac05-7b35195473a3" (UID: "d975aa47-763c-4684-ac05-7b35195473a3"). InnerVolumeSpecName "kube-api-access-slm4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:19:04.445415 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.445328    2564 generic.go:356] "Generic (PLEG): container finished" podID="d975aa47-763c-4684-ac05-7b35195473a3" containerID="dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323" exitCode=0
Sep 11 13:19:04.445415 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.445389    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"
Sep 11 13:19:04.445660 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.445407    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6" event={"ID":"d975aa47-763c-4684-ac05-7b35195473a3","Type":"ContainerDied","Data":"dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323"}
Sep 11 13:19:04.445660 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.445465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6" event={"ID":"d975aa47-763c-4684-ac05-7b35195473a3","Type":"ContainerDied","Data":"d5ceeb1bc2287f517dcc4ecf97c41580d1a412372368ffb9a06e0156e8f79c89"}
Sep 11 13:19:04.445660 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.445497    2564 scope.go:117] "RemoveContainer" containerID="dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323"
Sep 11 13:19:04.447183 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.447156    2564 generic.go:356] "Generic (PLEG): container finished" podID="4b763dfe-4ac4-4253-9696-6e7b4c218efa" containerID="dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a" exitCode=0
Sep 11 13:19:04.447307 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.447191    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"
Sep 11 13:19:04.447307 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.447208    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg" event={"ID":"4b763dfe-4ac4-4253-9696-6e7b4c218efa","Type":"ContainerDied","Data":"dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a"}
Sep 11 13:19:04.447307 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.447253    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg" event={"ID":"4b763dfe-4ac4-4253-9696-6e7b4c218efa","Type":"ContainerDied","Data":"a5abc6d9729dd4a57632c92910820332513743de05daab8471511a297d935e82"}
Sep 11 13:19:04.456767 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.456744    2564 scope.go:117] "RemoveContainer" containerID="dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323"
Sep 11 13:19:04.457147 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:19:04.457116    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323\": container with ID starting with dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323 not found: ID does not exist" containerID="dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323"
Sep 11 13:19:04.457280 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.457154    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323"} err="failed to get container status \"dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323\": rpc error: code = NotFound desc = could not find container \"dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323\": container with ID starting with dfc9d2e4de300289ded04322e56d6c30bccca8c96710297d6db0b881b4e9c323 not found: ID does not exist"
Sep 11 13:19:04.457280 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.457171    2564 scope.go:117] "RemoveContainer" containerID="dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a"
Sep 11 13:19:04.466854 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.466834    2564 scope.go:117] "RemoveContainer" containerID="dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a"
Sep 11 13:19:04.467154 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:19:04.467135    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a\": container with ID starting with dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a not found: ID does not exist" containerID="dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a"
Sep 11 13:19:04.467218 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.467168    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a"} err="failed to get container status \"dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a\": rpc error: code = NotFound desc = could not find container \"dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a\": container with ID starting with dc48971b0a9999ef18416a88142cd2153bd751859dfc288189c97c2c6feb919a not found: ID does not exist"
Sep 11 13:19:04.475553 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.475524    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"]
Sep 11 13:19:04.484150 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.484111    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d975aa47-763c-4684-ac05-7b35195473a3-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:19:04.484293 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.484158    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slm4v\" (UniqueName: \"kubernetes.io/projected/d975aa47-763c-4684-ac05-7b35195473a3-kube-api-access-slm4v\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:19:04.489816 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.489791    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-37c1cf-85659776cd-wqpxg"]
Sep 11 13:19:04.502103 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.502074    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"]
Sep 11 13:19:04.505971 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:04.505938    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-37c1cf-6d8c5f5855-hctd6"]
Sep 11 13:19:06.483829 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:06.483794    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b763dfe-4ac4-4253-9696-6e7b4c218efa" path="/var/lib/kubelet/pods/4b763dfe-4ac4-4253-9696-6e7b4c218efa/volumes"
Sep 11 13:19:06.484212 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:19:06.484176    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d975aa47-763c-4684-ac05-7b35195473a3" path="/var/lib/kubelet/pods/d975aa47-763c-4684-ac05-7b35195473a3/volumes"
Sep 11 13:20:04.414662 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.414550    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"]
Sep 11 13:20:04.415267 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.414824    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l" podUID="e16f4e99-6159-42d5-acfb-ce99e52c69cd" containerName="maas-api" containerID="cri-o://2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e" gracePeriod=30
Sep 11 13:20:04.693096 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.693060    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:20:04.773225 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.773185    2564 generic.go:356] "Generic (PLEG): container finished" podID="e16f4e99-6159-42d5-acfb-ce99e52c69cd" containerID="2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e" exitCode=0
Sep 11 13:20:04.773404 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.773236    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"
Sep 11 13:20:04.773404 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.773270    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l" event={"ID":"e16f4e99-6159-42d5-acfb-ce99e52c69cd","Type":"ContainerDied","Data":"2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e"}
Sep 11 13:20:04.773404 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.773322    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l" event={"ID":"e16f4e99-6159-42d5-acfb-ce99e52c69cd","Type":"ContainerDied","Data":"61df95fee010f76e96a14064d7521159151e93756e54890567b63d7ab010be99"}
Sep 11 13:20:04.773404 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.773342    2564 scope.go:117] "RemoveContainer" containerID="2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e"
Sep 11 13:20:04.782851 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.782822    2564 scope.go:117] "RemoveContainer" containerID="2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e"
Sep 11 13:20:04.783212 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:20:04.783188    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e\": container with ID starting with 2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e not found: ID does not exist" containerID="2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e"
Sep 11 13:20:04.783316 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.783222    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e"} err="failed to get container status \"2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e\": rpc error: code = NotFound desc = could not find container \"2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e\": container with ID starting with 2e349045c13c98ee7f54eb605ab81e31f527d360f8bc28ed3d55492ec5f76a5e not found: ID does not exist"
Sep 11 13:20:04.861126 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.861090    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l2k5\" (UniqueName: \"kubernetes.io/projected/e16f4e99-6159-42d5-acfb-ce99e52c69cd-kube-api-access-2l2k5\") pod \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") "
Sep 11 13:20:04.861331 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.861154    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-maas-api-tls\") pod \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") "
Sep 11 13:20:04.861331 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.861248    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e16f4e99-6159-42d5-acfb-ce99e52c69cd-openshift-service-ca\") pod \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") "
Sep 11 13:20:04.861331 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.861267    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-metrics-certs\") pod \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\" (UID: \"e16f4e99-6159-42d5-acfb-ce99e52c69cd\") "
Sep 11 13:20:04.861617 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.861586    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e16f4e99-6159-42d5-acfb-ce99e52c69cd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e16f4e99-6159-42d5-acfb-ce99e52c69cd" (UID: "e16f4e99-6159-42d5-acfb-ce99e52c69cd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:04.863470 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.863439    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e16f4e99-6159-42d5-acfb-ce99e52c69cd" (UID: "e16f4e99-6159-42d5-acfb-ce99e52c69cd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:04.863470 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.863454    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e16f4e99-6159-42d5-acfb-ce99e52c69cd" (UID: "e16f4e99-6159-42d5-acfb-ce99e52c69cd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:04.863630 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.863458    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e16f4e99-6159-42d5-acfb-ce99e52c69cd-kube-api-access-2l2k5" (OuterVolumeSpecName: "kube-api-access-2l2k5") pod "e16f4e99-6159-42d5-acfb-ce99e52c69cd" (UID: "e16f4e99-6159-42d5-acfb-ce99e52c69cd"). InnerVolumeSpecName "kube-api-access-2l2k5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:04.962633 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.962537    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:04.962633 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.962570    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e16f4e99-6159-42d5-acfb-ce99e52c69cd-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:04.962633 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.962582    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e16f4e99-6159-42d5-acfb-ce99e52c69cd-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:04.962633 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:04.962596    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l2k5\" (UniqueName: \"kubernetes.io/projected/e16f4e99-6159-42d5-acfb-ce99e52c69cd-kube-api-access-2l2k5\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:05.100703 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:05.100669    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"]
Sep 11 13:20:05.102633 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:05.102607    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-06286a-7885d69d65-6fs8l"]
Sep 11 13:20:06.484112 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:06.484064    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e16f4e99-6159-42d5-acfb-ce99e52c69cd" path="/var/lib/kubelet/pods/e16f4e99-6159-42d5-acfb-ce99e52c69cd/volumes"
Sep 11 13:20:22.447488 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.447447    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"]
Sep 11 13:20:22.448039 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.447801    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq" podUID="39695453-9282-4f0f-9279-83c1b1070a9a" containerName="maas-api" containerID="cri-o://88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067" gracePeriod=30
Sep 11 13:20:22.510414 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.510368    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"]
Sep 11 13:20:22.510677 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.510623    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd" podUID="9dc7aa67-7c68-4251-a339-cc5aa8a69792" containerName="payload-pre-processing" containerID="cri-o://df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903" gracePeriod=30
Sep 11 13:20:22.612484 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.612423    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq" podUID="39695453-9282-4f0f-9279-83c1b1070a9a" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.62:8443/health\": dial tcp 10.133.0.62:8443: connect: connection refused"
Sep 11 13:20:22.774744 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.774704    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:20:22.814954 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.814926    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:20:22.836426 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.836390    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39695453-9282-4f0f-9279-83c1b1070a9a-openshift-service-ca\") pod \"39695453-9282-4f0f-9279-83c1b1070a9a\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") "
Sep 11 13:20:22.836633 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.836452    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssc2t\" (UniqueName: \"kubernetes.io/projected/39695453-9282-4f0f-9279-83c1b1070a9a-kube-api-access-ssc2t\") pod \"39695453-9282-4f0f-9279-83c1b1070a9a\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") "
Sep 11 13:20:22.836633 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.836486    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-metrics-certs\") pod \"39695453-9282-4f0f-9279-83c1b1070a9a\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") "
Sep 11 13:20:22.836633 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.836575    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-maas-api-tls\") pod \"39695453-9282-4f0f-9279-83c1b1070a9a\" (UID: \"39695453-9282-4f0f-9279-83c1b1070a9a\") "
Sep 11 13:20:22.837247 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.837211    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39695453-9282-4f0f-9279-83c1b1070a9a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39695453-9282-4f0f-9279-83c1b1070a9a" (UID: "39695453-9282-4f0f-9279-83c1b1070a9a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:22.839609 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.839542    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "39695453-9282-4f0f-9279-83c1b1070a9a" (UID: "39695453-9282-4f0f-9279-83c1b1070a9a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:22.839785 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.839706    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39695453-9282-4f0f-9279-83c1b1070a9a-kube-api-access-ssc2t" (OuterVolumeSpecName: "kube-api-access-ssc2t") pod "39695453-9282-4f0f-9279-83c1b1070a9a" (UID: "39695453-9282-4f0f-9279-83c1b1070a9a"). InnerVolumeSpecName "kube-api-access-ssc2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:22.840764 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.840578    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39695453-9282-4f0f-9279-83c1b1070a9a" (UID: "39695453-9282-4f0f-9279-83c1b1070a9a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:22.877854 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.877819    2564 generic.go:356] "Generic (PLEG): container finished" podID="39695453-9282-4f0f-9279-83c1b1070a9a" containerID="88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067" exitCode=0
Sep 11 13:20:22.878066 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.877864    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"
Sep 11 13:20:22.878066 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.877907    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq" event={"ID":"39695453-9282-4f0f-9279-83c1b1070a9a","Type":"ContainerDied","Data":"88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067"}
Sep 11 13:20:22.878066 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.877968    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq" event={"ID":"39695453-9282-4f0f-9279-83c1b1070a9a","Type":"ContainerDied","Data":"4aafcc901666f53fd429f81d4bd6bdfd452c3451b9de94909d293508de360bf1"}
Sep 11 13:20:22.878066 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.878001    2564 scope.go:117] "RemoveContainer" containerID="88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067"
Sep 11 13:20:22.879685 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.879652    2564 generic.go:356] "Generic (PLEG): container finished" podID="9dc7aa67-7c68-4251-a339-cc5aa8a69792" containerID="df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903" exitCode=0
Sep 11 13:20:22.879802 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.879702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd" event={"ID":"9dc7aa67-7c68-4251-a339-cc5aa8a69792","Type":"ContainerDied","Data":"df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903"}
Sep 11 13:20:22.879802 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.879721    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"
Sep 11 13:20:22.879802 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.879742    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd" event={"ID":"9dc7aa67-7c68-4251-a339-cc5aa8a69792","Type":"ContainerDied","Data":"4682c6c50cc40dd6aa5bd5e3bf92baeb294a463eea4c3d3fda7765749ac209a1"}
Sep 11 13:20:22.888356 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.888337    2564 scope.go:117] "RemoveContainer" containerID="88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067"
Sep 11 13:20:22.888746 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:20:22.888723    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067\": container with ID starting with 88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067 not found: ID does not exist" containerID="88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067"
Sep 11 13:20:22.888862 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.888752    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067"} err="failed to get container status \"88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067\": rpc error: code = NotFound desc = could not find container \"88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067\": container with ID starting with 88e9986e7f6a17b9b91ff66cb085e9141e88a6c8e95082c76e5bdeea90065067 not found: ID does not exist"
Sep 11 13:20:22.888862 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.888770    2564 scope.go:117] "RemoveContainer" containerID="df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903"
Sep 11 13:20:22.897602 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.897578    2564 scope.go:117] "RemoveContainer" containerID="df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903"
Sep 11 13:20:22.897951 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:20:22.897926    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903\": container with ID starting with df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903 not found: ID does not exist" containerID="df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903"
Sep 11 13:20:22.898012 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.897963    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903"} err="failed to get container status \"df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903\": rpc error: code = NotFound desc = could not find container \"df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903\": container with ID starting with df8eea85d9d4b88ddea66db93a1a77e1013af07f5c720405f1c3b4407dbe0903 not found: ID does not exist"
Sep 11 13:20:22.904878 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.904844    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"]
Sep 11 13:20:22.908257 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.908226    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d4b0b9-59954cf695-k5xtq"]
Sep 11 13:20:22.937591 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.937557    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dc7aa67-7c68-4251-a339-cc5aa8a69792-plugins-config-volume\") pod \"9dc7aa67-7c68-4251-a339-cc5aa8a69792\" (UID: \"9dc7aa67-7c68-4251-a339-cc5aa8a69792\") "
Sep 11 13:20:22.937763 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.937610    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhmhf\" (UniqueName: \"kubernetes.io/projected/9dc7aa67-7c68-4251-a339-cc5aa8a69792-kube-api-access-qhmhf\") pod \"9dc7aa67-7c68-4251-a339-cc5aa8a69792\" (UID: \"9dc7aa67-7c68-4251-a339-cc5aa8a69792\") "
Sep 11 13:20:22.937968 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.937947    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.937968 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.937968    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39695453-9282-4f0f-9279-83c1b1070a9a-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.938138 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.937982    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssc2t\" (UniqueName: \"kubernetes.io/projected/39695453-9282-4f0f-9279-83c1b1070a9a-kube-api-access-ssc2t\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.938138 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.938005    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39695453-9282-4f0f-9279-83c1b1070a9a-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.938138 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.937976    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dc7aa67-7c68-4251-a339-cc5aa8a69792-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9dc7aa67-7c68-4251-a339-cc5aa8a69792" (UID: "9dc7aa67-7c68-4251-a339-cc5aa8a69792"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:22.939751 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:22.939720    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dc7aa67-7c68-4251-a339-cc5aa8a69792-kube-api-access-qhmhf" (OuterVolumeSpecName: "kube-api-access-qhmhf") pod "9dc7aa67-7c68-4251-a339-cc5aa8a69792" (UID: "9dc7aa67-7c68-4251-a339-cc5aa8a69792"). InnerVolumeSpecName "kube-api-access-qhmhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:23.039204 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:23.039109    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dc7aa67-7c68-4251-a339-cc5aa8a69792-plugins-config-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:23.039204 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:23.039142    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhmhf\" (UniqueName: \"kubernetes.io/projected/9dc7aa67-7c68-4251-a339-cc5aa8a69792-kube-api-access-qhmhf\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:23.204564 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:23.204518    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"]
Sep 11 13:20:23.206578 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:23.206551    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d4b0b9-7466cbdc94-xlsqd"]
Sep 11 13:20:24.483731 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:24.483697    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39695453-9282-4f0f-9279-83c1b1070a9a" path="/var/lib/kubelet/pods/39695453-9282-4f0f-9279-83c1b1070a9a/volumes"
Sep 11 13:20:24.484113 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:24.484071    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dc7aa67-7c68-4251-a339-cc5aa8a69792" path="/var/lib/kubelet/pods/9dc7aa67-7c68-4251-a339-cc5aa8a69792/volumes"
Sep 11 13:20:40.828681 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:40.828630    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"]
Sep 11 13:20:40.829118 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:40.828853    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb" podUID="4c5caac5-88ad-480a-95d9-ee33cfc354f0" containerName="maas-api" containerID="cri-o://3d7fecc43808ad05194b68519c5793a7640ee262bb9027c618941353be0cd33c" gracePeriod=30
Sep 11 13:20:40.986847 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:40.986697    2564 generic.go:356] "Generic (PLEG): container finished" podID="4c5caac5-88ad-480a-95d9-ee33cfc354f0" containerID="3d7fecc43808ad05194b68519c5793a7640ee262bb9027c618941353be0cd33c" exitCode=0
Sep 11 13:20:40.986847 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:40.986783    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb" event={"ID":"4c5caac5-88ad-480a-95d9-ee33cfc354f0","Type":"ContainerDied","Data":"3d7fecc43808ad05194b68519c5793a7640ee262bb9027c618941353be0cd33c"}
Sep 11 13:20:41.098499 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.098473    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:20:41.210400 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.210356    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c5caac5-88ad-480a-95d9-ee33cfc354f0-openshift-service-ca\") pod \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") "
Sep 11 13:20:41.210603 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.210462    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-maas-api-tls\") pod \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") "
Sep 11 13:20:41.210603 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.210556    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-metrics-certs\") pod \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") "
Sep 11 13:20:41.210783 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.210623    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vd4f\" (UniqueName: \"kubernetes.io/projected/4c5caac5-88ad-480a-95d9-ee33cfc354f0-kube-api-access-6vd4f\") pod \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\" (UID: \"4c5caac5-88ad-480a-95d9-ee33cfc354f0\") "
Sep 11 13:20:41.210887 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.210856    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c5caac5-88ad-480a-95d9-ee33cfc354f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c5caac5-88ad-480a-95d9-ee33cfc354f0" (UID: "4c5caac5-88ad-480a-95d9-ee33cfc354f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:41.211138 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.211114    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c5caac5-88ad-480a-95d9-ee33cfc354f0-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:41.212779 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.212748    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c5caac5-88ad-480a-95d9-ee33cfc354f0" (UID: "4c5caac5-88ad-480a-95d9-ee33cfc354f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:41.212884 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.212798    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4c5caac5-88ad-480a-95d9-ee33cfc354f0" (UID: "4c5caac5-88ad-480a-95d9-ee33cfc354f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:41.212884 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.212854    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c5caac5-88ad-480a-95d9-ee33cfc354f0-kube-api-access-6vd4f" (OuterVolumeSpecName: "kube-api-access-6vd4f") pod "4c5caac5-88ad-480a-95d9-ee33cfc354f0" (UID: "4c5caac5-88ad-480a-95d9-ee33cfc354f0"). InnerVolumeSpecName "kube-api-access-6vd4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:41.312403 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.312364    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:41.312403 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.312397    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vd4f\" (UniqueName: \"kubernetes.io/projected/4c5caac5-88ad-480a-95d9-ee33cfc354f0-kube-api-access-6vd4f\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:41.312403 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.312409    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c5caac5-88ad-480a-95d9-ee33cfc354f0-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:41.993842 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.993803    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb" event={"ID":"4c5caac5-88ad-480a-95d9-ee33cfc354f0","Type":"ContainerDied","Data":"d1379a736728762c32d74d3f296e7a2bd66a854f8e68347bd0eb266dff537978"}
Sep 11 13:20:41.993842 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.993826    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"
Sep 11 13:20:41.994405 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:41.993888    2564 scope.go:117] "RemoveContainer" containerID="3d7fecc43808ad05194b68519c5793a7640ee262bb9027c618941353be0cd33c"
Sep 11 13:20:42.020202 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:42.020165    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"]
Sep 11 13:20:42.027816 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:42.027783    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-2e7f38-947f5cfd4-9w4nb"]
Sep 11 13:20:42.485250 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:42.485216    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c5caac5-88ad-480a-95d9-ee33cfc354f0" path="/var/lib/kubelet/pods/4c5caac5-88ad-480a-95d9-ee33cfc354f0/volumes"
Sep 11 13:20:51.616301 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:51.616258    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-758b8949b9-2qctg"]
Sep 11 13:20:52.046377 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.046270    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-758b8949b9-2qctg" podUID="bb3a5593-6320-4701-b33f-93ecd7aea9fe" containerName="manager" containerID="cri-o://4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0" gracePeriod=30
Sep 11 13:20:52.303276 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.303208    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 13:20:52.414828 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.414796    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdfhz\" (UniqueName: \"kubernetes.io/projected/bb3a5593-6320-4701-b33f-93ecd7aea9fe-kube-api-access-sdfhz\") pod \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") "
Sep 11 13:20:52.415002 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.414842    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-metrics-certs\") pod \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") "
Sep 11 13:20:52.415002 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.414923    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-webhook-cert\") pod \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\" (UID: \"bb3a5593-6320-4701-b33f-93ecd7aea9fe\") "
Sep 11 13:20:52.416889 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.416856    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bb3a5593-6320-4701-b33f-93ecd7aea9fe" (UID: "bb3a5593-6320-4701-b33f-93ecd7aea9fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:52.417007 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.416912    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb3a5593-6320-4701-b33f-93ecd7aea9fe-kube-api-access-sdfhz" (OuterVolumeSpecName: "kube-api-access-sdfhz") pod "bb3a5593-6320-4701-b33f-93ecd7aea9fe" (UID: "bb3a5593-6320-4701-b33f-93ecd7aea9fe"). InnerVolumeSpecName "kube-api-access-sdfhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:52.417007 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.416916    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bb3a5593-6320-4701-b33f-93ecd7aea9fe" (UID: "bb3a5593-6320-4701-b33f-93ecd7aea9fe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:52.515916 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.515881    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdfhz\" (UniqueName: \"kubernetes.io/projected/bb3a5593-6320-4701-b33f-93ecd7aea9fe-kube-api-access-sdfhz\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:52.515916 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.515913    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:52.515916 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:52.515922    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb3a5593-6320-4701-b33f-93ecd7aea9fe-webhook-cert\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:20:53.054102 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:53.054066    2564 generic.go:356] "Generic (PLEG): container finished" podID="bb3a5593-6320-4701-b33f-93ecd7aea9fe" containerID="4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0" exitCode=0
Sep 11 13:20:53.054619 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:53.054123    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758b8949b9-2qctg"
Sep 11 13:20:53.054619 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:53.054161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758b8949b9-2qctg" event={"ID":"bb3a5593-6320-4701-b33f-93ecd7aea9fe","Type":"ContainerDied","Data":"4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0"}
Sep 11 13:20:53.054619 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:53.054262    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758b8949b9-2qctg" event={"ID":"bb3a5593-6320-4701-b33f-93ecd7aea9fe","Type":"ContainerDied","Data":"02da4ba6d0e461c4ec7e5d38ac1bfda2fb2792cf081b9430594571a36168cfe0"}
Sep 11 13:20:53.054619 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:53.054298    2564 scope.go:117] "RemoveContainer" containerID="4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0"
Sep 11 13:20:53.063898 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:53.063879    2564 scope.go:117] "RemoveContainer" containerID="4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0"
Sep 11 13:20:53.064208 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:20:53.064188    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0\": container with ID starting with 4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0 not found: ID does not exist" containerID="4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0"
Sep 11 13:20:53.064271 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:53.064221    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0"} err="failed to get container status \"4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0\": rpc error: code = NotFound desc = could not find container \"4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0\": container with ID starting with 4f6ac65421da6fb31ebdc7226ee010214c4594342bea2b47cc705bfd6f13c0d0 not found: ID does not exist"
Sep 11 13:20:53.073430 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:53.073387    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-758b8949b9-2qctg"]
Sep 11 13:20:53.078159 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:53.078125    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-758b8949b9-2qctg"]
Sep 11 13:20:54.483741 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:20:54.483702    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb3a5593-6320-4701-b33f-93ecd7aea9fe" path="/var/lib/kubelet/pods/bb3a5593-6320-4701-b33f-93ecd7aea9fe/volumes"
Sep 11 13:21:53.933230 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933191    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-758b8949b9-lwn7n"]
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933581    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b763dfe-4ac4-4253-9696-6e7b4c218efa" containerName="maas-api"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933592    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b763dfe-4ac4-4253-9696-6e7b4c218efa" containerName="maas-api"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933602    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c5caac5-88ad-480a-95d9-ee33cfc354f0" containerName="maas-api"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933607    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c5caac5-88ad-480a-95d9-ee33cfc354f0" containerName="maas-api"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933624    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d975aa47-763c-4684-ac05-7b35195473a3" containerName="payload-processing"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933630    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d975aa47-763c-4684-ac05-7b35195473a3" containerName="payload-processing"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933652    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f16b97-a816-480c-adce-d097778befac" containerName="payload-processing"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933661    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f16b97-a816-480c-adce-d097778befac" containerName="payload-processing"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933681    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dc7aa67-7c68-4251-a339-cc5aa8a69792" containerName="payload-pre-processing"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933689    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dc7aa67-7c68-4251-a339-cc5aa8a69792" containerName="payload-pre-processing"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933705    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e16f4e99-6159-42d5-acfb-ce99e52c69cd" containerName="maas-api"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933714    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e16f4e99-6159-42d5-acfb-ce99e52c69cd" containerName="maas-api"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933726    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39695453-9282-4f0f-9279-83c1b1070a9a" containerName="maas-api"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933733    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39695453-9282-4f0f-9279-83c1b1070a9a" containerName="maas-api"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933745    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb3a5593-6320-4701-b33f-93ecd7aea9fe" containerName="manager"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933752    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb3a5593-6320-4701-b33f-93ecd7aea9fe" containerName="manager"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933759    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da2891fc-7b28-4e71-a27a-9258cde2e185" containerName="payload-pre-processing"
Sep 11 13:21:53.933773 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933765    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="da2891fc-7b28-4e71-a27a-9258cde2e185" containerName="payload-pre-processing"
Sep 11 13:21:53.934335 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933835    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c5caac5-88ad-480a-95d9-ee33cfc354f0" containerName="maas-api"
Sep 11 13:21:53.934335 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933843    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39695453-9282-4f0f-9279-83c1b1070a9a" containerName="maas-api"
Sep 11 13:21:53.934335 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933850    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e16f4e99-6159-42d5-acfb-ce99e52c69cd" containerName="maas-api"
Sep 11 13:21:53.934335 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933856    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dc7aa67-7c68-4251-a339-cc5aa8a69792" containerName="payload-pre-processing"
Sep 11 13:21:53.934335 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933861    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="da2891fc-7b28-4e71-a27a-9258cde2e185" containerName="payload-pre-processing"
Sep 11 13:21:53.934335 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933866    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f16b97-a816-480c-adce-d097778befac" containerName="payload-processing"
Sep 11 13:21:53.934335 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933871    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d975aa47-763c-4684-ac05-7b35195473a3" containerName="payload-processing"
Sep 11 13:21:53.934335 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933877    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb3a5593-6320-4701-b33f-93ecd7aea9fe" containerName="manager"
Sep 11 13:21:53.934335 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.933881    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b763dfe-4ac4-4253-9696-6e7b4c218efa" containerName="maas-api"
Sep 11 13:21:53.936786 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.936767    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:53.939842 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.939817    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 13:21:53.940014 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.939820    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 13:21:53.940919 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.940898    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 13:21:53.941043 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.940987    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kxqtj\""
Sep 11 13:21:53.968748 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.968692    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-758b8949b9-lwn7n"]
Sep 11 13:21:53.980840 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.980797    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-metrics-certs\") pod \"maas-controller-758b8949b9-lwn7n\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") " pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:53.981000 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.980858    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr89p\" (UniqueName: \"kubernetes.io/projected/675802a6-03af-45e4-b830-887efafe7e92-kube-api-access-xr89p\") pod \"maas-controller-758b8949b9-lwn7n\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") " pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:53.981000 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:53.980966    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-webhook-cert\") pod \"maas-controller-758b8949b9-lwn7n\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") " pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:54.082378 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:54.082336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-webhook-cert\") pod \"maas-controller-758b8949b9-lwn7n\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") " pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:54.082575 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:54.082401    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-metrics-certs\") pod \"maas-controller-758b8949b9-lwn7n\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") " pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:54.082575 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:54.082449    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr89p\" (UniqueName: \"kubernetes.io/projected/675802a6-03af-45e4-b830-887efafe7e92-kube-api-access-xr89p\") pod \"maas-controller-758b8949b9-lwn7n\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") " pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:54.085067 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:54.085035    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-webhook-cert\") pod \"maas-controller-758b8949b9-lwn7n\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") " pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:54.085196 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:54.085035    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-metrics-certs\") pod \"maas-controller-758b8949b9-lwn7n\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") " pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:54.093483 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:54.093456    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr89p\" (UniqueName: \"kubernetes.io/projected/675802a6-03af-45e4-b830-887efafe7e92-kube-api-access-xr89p\") pod \"maas-controller-758b8949b9-lwn7n\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") " pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:54.247401 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:54.247299    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:54.546571 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:54.546542    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-758b8949b9-lwn7n"]
Sep 11 13:21:54.548239 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:21:54.548209    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod675802a6_03af_45e4_b830_887efafe7e92.slice/crio-d60b2c0ddef42c1e4b01814ef50aedefaf1d8fee12f30686d7ec8a478b82230b WatchSource:0}: Error finding container d60b2c0ddef42c1e4b01814ef50aedefaf1d8fee12f30686d7ec8a478b82230b: Status 404 returned error can't find the container with id d60b2c0ddef42c1e4b01814ef50aedefaf1d8fee12f30686d7ec8a478b82230b
Sep 11 13:21:54.549525 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:54.549508    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:21:55.380225 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:55.380128    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758b8949b9-lwn7n" event={"ID":"675802a6-03af-45e4-b830-887efafe7e92","Type":"ContainerStarted","Data":"1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9"}
Sep 11 13:21:55.380225 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:55.380185    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758b8949b9-lwn7n" event={"ID":"675802a6-03af-45e4-b830-887efafe7e92","Type":"ContainerStarted","Data":"d60b2c0ddef42c1e4b01814ef50aedefaf1d8fee12f30686d7ec8a478b82230b"}
Sep 11 13:21:55.380771 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:55.380274    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:21:55.407428 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:21:55.407369    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-758b8949b9-lwn7n" podStartSLOduration=1.845092412 podStartE2EDuration="2.407352798s" totalImagesPullingTime="562.260386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:55.40653757 +0000 UTC m=+2083.468844528" watchObservedRunningTime="2026-09-11 13:21:55.407352798 +0000 UTC m=+2083.469659755"
Sep 11 13:22:06.387342 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:06.387310    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:22:12.614660 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:12.614618    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:22:12.620429 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:12.620400    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:22:15.836214 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:15.836169    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-758b8949b9-lwn7n"]
Sep 11 13:22:15.836747 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:15.836447    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-758b8949b9-lwn7n" podUID="675802a6-03af-45e4-b830-887efafe7e92" containerName="manager" containerID="cri-o://1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9" gracePeriod=30
Sep 11 13:22:16.091249 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.091181    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:22:16.185174 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.185140    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-webhook-cert\") pod \"675802a6-03af-45e4-b830-887efafe7e92\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") "
Sep 11 13:22:16.185362 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.185253    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr89p\" (UniqueName: \"kubernetes.io/projected/675802a6-03af-45e4-b830-887efafe7e92-kube-api-access-xr89p\") pod \"675802a6-03af-45e4-b830-887efafe7e92\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") "
Sep 11 13:22:16.185362 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.185277    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-metrics-certs\") pod \"675802a6-03af-45e4-b830-887efafe7e92\" (UID: \"675802a6-03af-45e4-b830-887efafe7e92\") "
Sep 11 13:22:16.187320 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.187283    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "675802a6-03af-45e4-b830-887efafe7e92" (UID: "675802a6-03af-45e4-b830-887efafe7e92"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:22:16.187449 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.187352    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "675802a6-03af-45e4-b830-887efafe7e92" (UID: "675802a6-03af-45e4-b830-887efafe7e92"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:22:16.187449 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.187391    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/675802a6-03af-45e4-b830-887efafe7e92-kube-api-access-xr89p" (OuterVolumeSpecName: "kube-api-access-xr89p") pod "675802a6-03af-45e4-b830-887efafe7e92" (UID: "675802a6-03af-45e4-b830-887efafe7e92"). InnerVolumeSpecName "kube-api-access-xr89p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:16.286691 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.286635    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-webhook-cert\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:22:16.286691 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.286691    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr89p\" (UniqueName: \"kubernetes.io/projected/675802a6-03af-45e4-b830-887efafe7e92-kube-api-access-xr89p\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:22:16.286897 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.286702    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/675802a6-03af-45e4-b830-887efafe7e92-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:22:16.493910 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.493882    2564 generic.go:356] "Generic (PLEG): container finished" podID="675802a6-03af-45e4-b830-887efafe7e92" containerID="1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9" exitCode=0
Sep 11 13:22:16.494064 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.493937    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758b8949b9-lwn7n"
Sep 11 13:22:16.494064 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.493964    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758b8949b9-lwn7n" event={"ID":"675802a6-03af-45e4-b830-887efafe7e92","Type":"ContainerDied","Data":"1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9"}
Sep 11 13:22:16.494064 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.494016    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758b8949b9-lwn7n" event={"ID":"675802a6-03af-45e4-b830-887efafe7e92","Type":"ContainerDied","Data":"d60b2c0ddef42c1e4b01814ef50aedefaf1d8fee12f30686d7ec8a478b82230b"}
Sep 11 13:22:16.494064 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.494039    2564 scope.go:117] "RemoveContainer" containerID="1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9"
Sep 11 13:22:16.503054 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.503033    2564 scope.go:117] "RemoveContainer" containerID="1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9"
Sep 11 13:22:16.503391 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:22:16.503370    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9\": container with ID starting with 1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9 not found: ID does not exist" containerID="1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9"
Sep 11 13:22:16.503440 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.503400    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9"} err="failed to get container status \"1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9\": rpc error: code = NotFound desc = could not find container \"1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9\": container with ID starting with 1b1383ffc1aba118f8c005a986d208c4a5cc4ea0be1f682bcafa0b2833189fb9 not found: ID does not exist"
Sep 11 13:22:16.521587 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.521557    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-758b8949b9-lwn7n"]
Sep 11 13:22:16.525554 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:16.525528    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-758b8949b9-lwn7n"]
Sep 11 13:22:17.700055 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.700017    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-758b8949b9-6njfb"]
Sep 11 13:22:17.700504 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.700489    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="675802a6-03af-45e4-b830-887efafe7e92" containerName="manager"
Sep 11 13:22:17.700549 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.700505    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="675802a6-03af-45e4-b830-887efafe7e92" containerName="manager"
Sep 11 13:22:17.700585 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.700572    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="675802a6-03af-45e4-b830-887efafe7e92" containerName="manager"
Sep 11 13:22:17.704990 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.704962    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:17.709441 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.709413    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 13:22:17.709585 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.709460    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 13:22:17.709585 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.709471    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 13:22:17.709585 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.709510    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kxqtj\""
Sep 11 13:22:17.718103 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.718070    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-758b8949b9-6njfb"]
Sep 11 13:22:17.803248 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.803209    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a76cbc95-05b1-415d-a2c0-db0770311311-webhook-cert\") pod \"maas-controller-758b8949b9-6njfb\" (UID: \"a76cbc95-05b1-415d-a2c0-db0770311311\") " pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:17.803439 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.803292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d52sr\" (UniqueName: \"kubernetes.io/projected/a76cbc95-05b1-415d-a2c0-db0770311311-kube-api-access-d52sr\") pod \"maas-controller-758b8949b9-6njfb\" (UID: \"a76cbc95-05b1-415d-a2c0-db0770311311\") " pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:17.803439 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.803393    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a76cbc95-05b1-415d-a2c0-db0770311311-metrics-certs\") pod \"maas-controller-758b8949b9-6njfb\" (UID: \"a76cbc95-05b1-415d-a2c0-db0770311311\") " pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:17.904706 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.904666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d52sr\" (UniqueName: \"kubernetes.io/projected/a76cbc95-05b1-415d-a2c0-db0770311311-kube-api-access-d52sr\") pod \"maas-controller-758b8949b9-6njfb\" (UID: \"a76cbc95-05b1-415d-a2c0-db0770311311\") " pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:17.904894 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.904737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a76cbc95-05b1-415d-a2c0-db0770311311-metrics-certs\") pod \"maas-controller-758b8949b9-6njfb\" (UID: \"a76cbc95-05b1-415d-a2c0-db0770311311\") " pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:17.904894 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.904779    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a76cbc95-05b1-415d-a2c0-db0770311311-webhook-cert\") pod \"maas-controller-758b8949b9-6njfb\" (UID: \"a76cbc95-05b1-415d-a2c0-db0770311311\") " pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:17.907449 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.907410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a76cbc95-05b1-415d-a2c0-db0770311311-metrics-certs\") pod \"maas-controller-758b8949b9-6njfb\" (UID: \"a76cbc95-05b1-415d-a2c0-db0770311311\") " pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:17.907577 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.907410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a76cbc95-05b1-415d-a2c0-db0770311311-webhook-cert\") pod \"maas-controller-758b8949b9-6njfb\" (UID: \"a76cbc95-05b1-415d-a2c0-db0770311311\") " pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:17.914706 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:17.914679    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d52sr\" (UniqueName: \"kubernetes.io/projected/a76cbc95-05b1-415d-a2c0-db0770311311-kube-api-access-d52sr\") pod \"maas-controller-758b8949b9-6njfb\" (UID: \"a76cbc95-05b1-415d-a2c0-db0770311311\") " pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:18.015869 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:18.015771    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:18.290996 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:18.290967    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-758b8949b9-6njfb"]
Sep 11 13:22:18.292731 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:22:18.292687    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda76cbc95_05b1_415d_a2c0_db0770311311.slice/crio-d2a4370d801661d5237a921a7abef2bd4e4a3e7f6f23426be7ae1ee5e156734a WatchSource:0}: Error finding container d2a4370d801661d5237a921a7abef2bd4e4a3e7f6f23426be7ae1ee5e156734a: Status 404 returned error can't find the container with id d2a4370d801661d5237a921a7abef2bd4e4a3e7f6f23426be7ae1ee5e156734a
Sep 11 13:22:18.483582 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:18.483546    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="675802a6-03af-45e4-b830-887efafe7e92" path="/var/lib/kubelet/pods/675802a6-03af-45e4-b830-887efafe7e92/volumes"
Sep 11 13:22:18.507591 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:18.507558    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758b8949b9-6njfb" event={"ID":"a76cbc95-05b1-415d-a2c0-db0770311311","Type":"ContainerStarted","Data":"d2a4370d801661d5237a921a7abef2bd4e4a3e7f6f23426be7ae1ee5e156734a"}
Sep 11 13:22:19.514532 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:19.514492    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-758b8949b9-6njfb" event={"ID":"a76cbc95-05b1-415d-a2c0-db0770311311","Type":"ContainerStarted","Data":"99e3429f85744238cb341a8143eba6cc2ada9419a6120012846191aa8eba59cb"}
Sep 11 13:22:19.514945 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:19.514610    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:22:19.538892 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:19.538837    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-758b8949b9-6njfb" podStartSLOduration=2.069268569 podStartE2EDuration="2.538818219s" totalImagesPullingTime="469.54965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:22:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:19.536027024 +0000 UTC m=+2107.598333985" watchObservedRunningTime="2026-09-11 13:22:19.538818219 +0000 UTC m=+2107.601125176"
Sep 11 13:22:30.522535 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:22:30.522497    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-758b8949b9-6njfb"
Sep 11 13:23:30.710827 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:30.710792    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"]
Sep 11 13:23:30.711303 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:30.711037    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8" podUID="341e7875-a1bd-4402-93db-6d4131cbaed3" containerName="manager" containerID="cri-o://fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985" gracePeriod=10
Sep 11 13:23:31.229339 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.229237    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8" podUID="341e7875-a1bd-4402-93db-6d4131cbaed3" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.37:8081/readyz\": dial tcp 10.133.0.37:8081: connect: connection refused"
Sep 11 13:23:31.561274 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.561247    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 13:23:31.616408 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.616373    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z4vm\" (UniqueName: \"kubernetes.io/projected/341e7875-a1bd-4402-93db-6d4131cbaed3-kube-api-access-4z4vm\") pod \"341e7875-a1bd-4402-93db-6d4131cbaed3\" (UID: \"341e7875-a1bd-4402-93db-6d4131cbaed3\") "
Sep 11 13:23:31.616609 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.616485    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/341e7875-a1bd-4402-93db-6d4131cbaed3-extensions-socket-volume\") pod \"341e7875-a1bd-4402-93db-6d4131cbaed3\" (UID: \"341e7875-a1bd-4402-93db-6d4131cbaed3\") "
Sep 11 13:23:31.617239 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.616888    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341e7875-a1bd-4402-93db-6d4131cbaed3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "341e7875-a1bd-4402-93db-6d4131cbaed3" (UID: "341e7875-a1bd-4402-93db-6d4131cbaed3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:31.618682 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.618653    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/341e7875-a1bd-4402-93db-6d4131cbaed3-kube-api-access-4z4vm" (OuterVolumeSpecName: "kube-api-access-4z4vm") pod "341e7875-a1bd-4402-93db-6d4131cbaed3" (UID: "341e7875-a1bd-4402-93db-6d4131cbaed3"). InnerVolumeSpecName "kube-api-access-4z4vm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:31.717305 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.717274    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z4vm\" (UniqueName: \"kubernetes.io/projected/341e7875-a1bd-4402-93db-6d4131cbaed3-kube-api-access-4z4vm\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:23:31.717305 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.717297    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/341e7875-a1bd-4402-93db-6d4131cbaed3-extensions-socket-volume\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:23:31.911420 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.911382    2564 generic.go:356] "Generic (PLEG): container finished" podID="341e7875-a1bd-4402-93db-6d4131cbaed3" containerID="fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985" exitCode=0
Sep 11 13:23:31.911593 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.911431    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"
Sep 11 13:23:31.911593 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.911452    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8" event={"ID":"341e7875-a1bd-4402-93db-6d4131cbaed3","Type":"ContainerDied","Data":"fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985"}
Sep 11 13:23:31.911593 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.911505    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8" event={"ID":"341e7875-a1bd-4402-93db-6d4131cbaed3","Type":"ContainerDied","Data":"343f9e4d9cc952eaa895177be0ccfbdd15ddb36e350a83d08d4d9173f1ecfafe"}
Sep 11 13:23:31.911593 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.911535    2564 scope.go:117] "RemoveContainer" containerID="fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985"
Sep 11 13:23:31.921611 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.921587    2564 scope.go:117] "RemoveContainer" containerID="fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985"
Sep 11 13:23:31.921973 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:23:31.921952    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985\": container with ID starting with fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985 not found: ID does not exist" containerID="fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985"
Sep 11 13:23:31.922017 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.921982    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985"} err="failed to get container status \"fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985\": rpc error: code = NotFound desc = could not find container \"fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985\": container with ID starting with fda72c9c29d13a75be7bc811daf2572fc9f26a979a4a4d26f2bb532d1b67f985 not found: ID does not exist"
Sep 11 13:23:31.936251 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.936223    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"]
Sep 11 13:23:31.940361 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:31.940335    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cwpj8"]
Sep 11 13:23:32.484432 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:23:32.484388    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="341e7875-a1bd-4402-93db-6d4131cbaed3" path="/var/lib/kubelet/pods/341e7875-a1bd-4402-93db-6d4131cbaed3/volumes"
Sep 11 13:24:31.510943 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.510864    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"]
Sep 11 13:24:31.511393 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.511298    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="341e7875-a1bd-4402-93db-6d4131cbaed3" containerName="manager"
Sep 11 13:24:31.511393 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.511311    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="341e7875-a1bd-4402-93db-6d4131cbaed3" containerName="manager"
Sep 11 13:24:31.511469 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.511395    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="341e7875-a1bd-4402-93db-6d4131cbaed3" containerName="manager"
Sep 11 13:24:31.514536 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.514515    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"
Sep 11 13:24:31.518076 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.518050    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7rmlw\""
Sep 11 13:24:31.535005 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.534971    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"]
Sep 11 13:24:31.664825 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.664788    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71158af1-0f29-445f-a7e4-6565e6dd1bc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cngsb\" (UID: \"71158af1-0f29-445f-a7e4-6565e6dd1bc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"
Sep 11 13:24:31.665007 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.664849    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gbtm\" (UniqueName: \"kubernetes.io/projected/71158af1-0f29-445f-a7e4-6565e6dd1bc7-kube-api-access-5gbtm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cngsb\" (UID: \"71158af1-0f29-445f-a7e4-6565e6dd1bc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"
Sep 11 13:24:31.766347 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.766247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71158af1-0f29-445f-a7e4-6565e6dd1bc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cngsb\" (UID: \"71158af1-0f29-445f-a7e4-6565e6dd1bc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"
Sep 11 13:24:31.766347 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.766313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gbtm\" (UniqueName: \"kubernetes.io/projected/71158af1-0f29-445f-a7e4-6565e6dd1bc7-kube-api-access-5gbtm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cngsb\" (UID: \"71158af1-0f29-445f-a7e4-6565e6dd1bc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"
Sep 11 13:24:31.766762 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.766739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71158af1-0f29-445f-a7e4-6565e6dd1bc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cngsb\" (UID: \"71158af1-0f29-445f-a7e4-6565e6dd1bc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"
Sep 11 13:24:31.794228 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.794196    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gbtm\" (UniqueName: \"kubernetes.io/projected/71158af1-0f29-445f-a7e4-6565e6dd1bc7-kube-api-access-5gbtm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cngsb\" (UID: \"71158af1-0f29-445f-a7e4-6565e6dd1bc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"
Sep 11 13:24:31.825071 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:31.825030    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"
Sep 11 13:24:32.129461 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:32.129433    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"]
Sep 11 13:24:32.132315 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:24:32.132288    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71158af1_0f29_445f_a7e4_6565e6dd1bc7.slice/crio-3253ba37b96b3415583b0ac32e4a0e8eecaed0efacfafb5a7f337f444cd2849c WatchSource:0}: Error finding container 3253ba37b96b3415583b0ac32e4a0e8eecaed0efacfafb5a7f337f444cd2849c: Status 404 returned error can't find the container with id 3253ba37b96b3415583b0ac32e4a0e8eecaed0efacfafb5a7f337f444cd2849c
Sep 11 13:24:32.229477 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:32.229434    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb" event={"ID":"71158af1-0f29-445f-a7e4-6565e6dd1bc7","Type":"ContainerStarted","Data":"e661d995da072b7050b66c28ba1888937467741e37c2ffcf37a250c342fbdabb"}
Sep 11 13:24:32.229687 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:32.229508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb" event={"ID":"71158af1-0f29-445f-a7e4-6565e6dd1bc7","Type":"ContainerStarted","Data":"3253ba37b96b3415583b0ac32e4a0e8eecaed0efacfafb5a7f337f444cd2849c"}
Sep 11 13:24:32.229687 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:32.229536    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"
Sep 11 13:24:32.279939 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:32.279879    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb" podStartSLOduration=1.279860657 podStartE2EDuration="1.279860657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:32.275098329 +0000 UTC m=+2240.337405288" watchObservedRunningTime="2026-09-11 13:24:32.279860657 +0000 UTC m=+2240.342167615"
Sep 11 13:24:43.237772 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:24:43.237737    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cngsb"
Sep 11 13:25:24.531033 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.530984    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"]
Sep 11 13:25:24.534897 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.534869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.540377 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.540350    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-c2ce48f8-kserve-self-signed-certs\""
Sep 11 13:25:24.554524 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.554486    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-home\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.554524 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.554527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-model-cache\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.554794 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.554556    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78m8q\" (UniqueName: \"kubernetes.io/projected/a711a0f7-3711-4217-b6ef-a3d08302335a-kube-api-access-78m8q\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.554794 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.554672    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-dshm\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.554794 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.554721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-tmp-dir\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.554901 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.554820    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a711a0f7-3711-4217-b6ef-a3d08302335a-tls-certs\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.646888 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.646852    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"]
Sep 11 13:25:24.656288 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.656250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a711a0f7-3711-4217-b6ef-a3d08302335a-tls-certs\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.656515 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.656323    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-home\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.656515 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.656355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-model-cache\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.656515 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.656385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78m8q\" (UniqueName: \"kubernetes.io/projected/a711a0f7-3711-4217-b6ef-a3d08302335a-kube-api-access-78m8q\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.656515 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.656429    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-dshm\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.656515 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.656480    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-tmp-dir\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.656864 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.656839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-home\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.656930 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.656868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-model-cache\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.656930 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.656925    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-tmp-dir\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.658805 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.658772    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-dshm\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.659068 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.659047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a711a0f7-3711-4217-b6ef-a3d08302335a-tls-certs\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.721319 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.721283    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78m8q\" (UniqueName: \"kubernetes.io/projected/a711a0f7-3711-4217-b6ef-a3d08302335a-kube-api-access-78m8q\") pod \"e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") " pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:24.846809 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:24.846769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:25.156548 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:25.156428    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"]
Sep 11 13:25:25.159401 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:25:25.159370    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda711a0f7_3711_4217_b6ef_a3d08302335a.slice/crio-68f9aff2569281c106fe8bcbea4a72ee04172540aa9c8a57f579636b48c0ce49 WatchSource:0}: Error finding container 68f9aff2569281c106fe8bcbea4a72ee04172540aa9c8a57f579636b48c0ce49: Status 404 returned error can't find the container with id 68f9aff2569281c106fe8bcbea4a72ee04172540aa9c8a57f579636b48c0ce49
Sep 11 13:25:25.515103 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:25.515014    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs" event={"ID":"a711a0f7-3711-4217-b6ef-a3d08302335a","Type":"ContainerStarted","Data":"9fb69913e897ef9d5bfd4324762edbea4f27c431632658c1a39ba349e263a65e"}
Sep 11 13:25:25.515103 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:25.515058    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs" event={"ID":"a711a0f7-3711-4217-b6ef-a3d08302335a","Type":"ContainerStarted","Data":"68f9aff2569281c106fe8bcbea4a72ee04172540aa9c8a57f579636b48c0ce49"}
Sep 11 13:25:25.540244 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:25.540179    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs" podStartSLOduration=1.540154386 podStartE2EDuration="1.540154386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:25.537529879 +0000 UTC m=+2293.599836837" watchObservedRunningTime="2026-09-11 13:25:25.540154386 +0000 UTC m=+2293.602461346"
Sep 11 13:25:34.847733 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:34.847692    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:34.848206 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:34.847758    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:34.861786 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:34.861758    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:35.580836 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:35.580800    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:25:59.593453 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:25:59.593420    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"]
Sep 11 13:26:14.613171 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.613073    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs" podUID="a711a0f7-3711-4217-b6ef-a3d08302335a" containerName="main" containerID="cri-o://9fb69913e897ef9d5bfd4324762edbea4f27c431632658c1a39ba349e263a65e" gracePeriod=45
Sep 11 13:26:14.787771 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.787735    2564 generic.go:356] "Generic (PLEG): container finished" podID="a711a0f7-3711-4217-b6ef-a3d08302335a" containerID="9fb69913e897ef9d5bfd4324762edbea4f27c431632658c1a39ba349e263a65e" exitCode=0
Sep 11 13:26:14.788046 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.787822    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs" event={"ID":"a711a0f7-3711-4217-b6ef-a3d08302335a","Type":"ContainerDied","Data":"9fb69913e897ef9d5bfd4324762edbea4f27c431632658c1a39ba349e263a65e"}
Sep 11 13:26:14.860988 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.860963    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:26:14.934235 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934140    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-model-cache\") pod \"a711a0f7-3711-4217-b6ef-a3d08302335a\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") "
Sep 11 13:26:14.934235 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934216    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78m8q\" (UniqueName: \"kubernetes.io/projected/a711a0f7-3711-4217-b6ef-a3d08302335a-kube-api-access-78m8q\") pod \"a711a0f7-3711-4217-b6ef-a3d08302335a\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") "
Sep 11 13:26:14.934456 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934252    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-tmp-dir\") pod \"a711a0f7-3711-4217-b6ef-a3d08302335a\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") "
Sep 11 13:26:14.934456 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934299    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a711a0f7-3711-4217-b6ef-a3d08302335a-tls-certs\") pod \"a711a0f7-3711-4217-b6ef-a3d08302335a\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") "
Sep 11 13:26:14.934456 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934349    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-home\") pod \"a711a0f7-3711-4217-b6ef-a3d08302335a\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") "
Sep 11 13:26:14.934456 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934381    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-dshm\") pod \"a711a0f7-3711-4217-b6ef-a3d08302335a\" (UID: \"a711a0f7-3711-4217-b6ef-a3d08302335a\") "
Sep 11 13:26:14.934691 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934445    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-model-cache" (OuterVolumeSpecName: "model-cache") pod "a711a0f7-3711-4217-b6ef-a3d08302335a" (UID: "a711a0f7-3711-4217-b6ef-a3d08302335a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:26:14.934691 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934526    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a711a0f7-3711-4217-b6ef-a3d08302335a" (UID: "a711a0f7-3711-4217-b6ef-a3d08302335a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:26:14.934691 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934671    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-home" (OuterVolumeSpecName: "home") pod "a711a0f7-3711-4217-b6ef-a3d08302335a" (UID: "a711a0f7-3711-4217-b6ef-a3d08302335a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:26:14.934825 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934727    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-model-cache\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:26:14.934825 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.934742    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-tmp-dir\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:26:14.936630 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.936604    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-dshm" (OuterVolumeSpecName: "dshm") pod "a711a0f7-3711-4217-b6ef-a3d08302335a" (UID: "a711a0f7-3711-4217-b6ef-a3d08302335a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:26:14.936630 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.936619    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a711a0f7-3711-4217-b6ef-a3d08302335a-kube-api-access-78m8q" (OuterVolumeSpecName: "kube-api-access-78m8q") pod "a711a0f7-3711-4217-b6ef-a3d08302335a" (UID: "a711a0f7-3711-4217-b6ef-a3d08302335a"). InnerVolumeSpecName "kube-api-access-78m8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:26:14.936630 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:14.936601    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a711a0f7-3711-4217-b6ef-a3d08302335a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a711a0f7-3711-4217-b6ef-a3d08302335a" (UID: "a711a0f7-3711-4217-b6ef-a3d08302335a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:26:15.035445 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:15.035406    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78m8q\" (UniqueName: \"kubernetes.io/projected/a711a0f7-3711-4217-b6ef-a3d08302335a-kube-api-access-78m8q\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:26:15.035445 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:15.035438    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a711a0f7-3711-4217-b6ef-a3d08302335a-tls-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:26:15.035445 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:15.035449    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-home\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:26:15.035445 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:15.035457    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a711a0f7-3711-4217-b6ef-a3d08302335a-dshm\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:26:15.795550 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:15.795515    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"
Sep 11 13:26:15.796036 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:15.795515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs" event={"ID":"a711a0f7-3711-4217-b6ef-a3d08302335a","Type":"ContainerDied","Data":"68f9aff2569281c106fe8bcbea4a72ee04172540aa9c8a57f579636b48c0ce49"}
Sep 11 13:26:15.796036 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:15.795658    2564 scope.go:117] "RemoveContainer" containerID="9fb69913e897ef9d5bfd4324762edbea4f27c431632658c1a39ba349e263a65e"
Sep 11 13:26:15.817602 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:15.817569    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"]
Sep 11 13:26:15.819333 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:15.819311    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-c2ce48f8-kserve-687c78d755-2xsfs"]
Sep 11 13:26:16.483947 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:16.483914    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a711a0f7-3711-4217-b6ef-a3d08302335a" path="/var/lib/kubelet/pods/a711a0f7-3711-4217-b6ef-a3d08302335a/volumes"
Sep 11 13:26:17.917601 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:17.917566    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"]
Sep 11 13:26:17.918020 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:17.918005    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a711a0f7-3711-4217-b6ef-a3d08302335a" containerName="main"
Sep 11 13:26:17.918020 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:17.918021    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a711a0f7-3711-4217-b6ef-a3d08302335a" containerName="main"
Sep 11 13:26:17.918110 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:17.918101    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a711a0f7-3711-4217-b6ef-a3d08302335a" containerName="main"
Sep 11 13:26:17.921199 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:17.921180    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:17.932247 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:17.932213    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"]
Sep 11 13:26:18.064806 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.064759    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szbf6\" (UniqueName: \"kubernetes.io/projected/647074c8-18b1-4285-8d02-5781a3c80e6f-kube-api-access-szbf6\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.065031 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.064818    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/647074c8-18b1-4285-8d02-5781a3c80e6f-metrics-certs\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.065031 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.064985    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/647074c8-18b1-4285-8d02-5781a3c80e6f-maas-api-tls\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.065163 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.065078    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/647074c8-18b1-4285-8d02-5781a3c80e6f-openshift-service-ca\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.165873 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.165834    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/647074c8-18b1-4285-8d02-5781a3c80e6f-maas-api-tls\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.166099 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.165904    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/647074c8-18b1-4285-8d02-5781a3c80e6f-openshift-service-ca\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.166099 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.165935    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szbf6\" (UniqueName: \"kubernetes.io/projected/647074c8-18b1-4285-8d02-5781a3c80e6f-kube-api-access-szbf6\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.166099 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.165959    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/647074c8-18b1-4285-8d02-5781a3c80e6f-metrics-certs\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.166769 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.166745    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/647074c8-18b1-4285-8d02-5781a3c80e6f-openshift-service-ca\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.168530 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.168469    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/647074c8-18b1-4285-8d02-5781a3c80e6f-metrics-certs\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.168635 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.168539    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/647074c8-18b1-4285-8d02-5781a3c80e6f-maas-api-tls\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.174087 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.174056    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szbf6\" (UniqueName: \"kubernetes.io/projected/647074c8-18b1-4285-8d02-5781a3c80e6f-kube-api-access-szbf6\") pod \"maas-api-7f866d8d77-86g5w\" (UID: \"647074c8-18b1-4285-8d02-5781a3c80e6f\") " pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.233191 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.233146    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:18.511621 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.511595    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"]
Sep 11 13:26:18.513592 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:26:18.513560    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod647074c8_18b1_4285_8d02_5781a3c80e6f.slice/crio-feddd00059f09eca79325ed6697e230679b732d4c3c17edd797975889a260a74 WatchSource:0}: Error finding container feddd00059f09eca79325ed6697e230679b732d4c3c17edd797975889a260a74: Status 404 returned error can't find the container with id feddd00059f09eca79325ed6697e230679b732d4c3c17edd797975889a260a74
Sep 11 13:26:18.813181 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:18.813090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w" event={"ID":"647074c8-18b1-4285-8d02-5781a3c80e6f","Type":"ContainerStarted","Data":"feddd00059f09eca79325ed6697e230679b732d4c3c17edd797975889a260a74"}
Sep 11 13:26:19.820079 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:19.820040    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w" event={"ID":"647074c8-18b1-4285-8d02-5781a3c80e6f","Type":"ContainerStarted","Data":"7a4fc8510b403d11cfd8e4e1c671f6d35ce89d4640d85e07d625995f75a8a267"}
Sep 11 13:26:19.820484 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:19.820102    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:19.839293 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:19.839247    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w" podStartSLOduration=2.37345897 podStartE2EDuration="2.839229847s" totalImagesPullingTime="465.770877ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:19.838319096 +0000 UTC m=+2347.900626055" watchObservedRunningTime="2026-09-11 13:26:19.839229847 +0000 UTC m=+2347.901536805"
Sep 11 13:26:25.831022 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:25.830989    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f866d8d77-86g5w"
Sep 11 13:26:25.895506 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:25.895468    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"]
Sep 11 13:26:25.895778 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:25.895750    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn" podUID="aa05ff05-d220-4d14-9374-ab47e05f5b6c" containerName="maas-api" containerID="cri-o://aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b" gracePeriod=30
Sep 11 13:26:26.866871 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:26.866824    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn" podUID="aa05ff05-d220-4d14-9374-ab47e05f5b6c" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.44:8443/health\": dial tcp 10.133.0.44:8443: connect: connection refused"
Sep 11 13:26:27.143364 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.143332    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:26:27.251199 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.251156    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-maas-api-tls\") pod \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") "
Sep 11 13:26:27.251390 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.251208    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpl2p\" (UniqueName: \"kubernetes.io/projected/aa05ff05-d220-4d14-9374-ab47e05f5b6c-kube-api-access-zpl2p\") pod \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") "
Sep 11 13:26:27.251390 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.251255    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa05ff05-d220-4d14-9374-ab47e05f5b6c-openshift-service-ca\") pod \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") "
Sep 11 13:26:27.251390 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.251314    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-metrics-certs\") pod \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\" (UID: \"aa05ff05-d220-4d14-9374-ab47e05f5b6c\") "
Sep 11 13:26:27.251758 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.251725    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa05ff05-d220-4d14-9374-ab47e05f5b6c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa05ff05-d220-4d14-9374-ab47e05f5b6c" (UID: "aa05ff05-d220-4d14-9374-ab47e05f5b6c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:26:27.253555 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.253524    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa05ff05-d220-4d14-9374-ab47e05f5b6c" (UID: "aa05ff05-d220-4d14-9374-ab47e05f5b6c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:26:27.253555 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.253523    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa05ff05-d220-4d14-9374-ab47e05f5b6c-kube-api-access-zpl2p" (OuterVolumeSpecName: "kube-api-access-zpl2p") pod "aa05ff05-d220-4d14-9374-ab47e05f5b6c" (UID: "aa05ff05-d220-4d14-9374-ab47e05f5b6c"). InnerVolumeSpecName "kube-api-access-zpl2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:26:27.253766 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.253678    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aa05ff05-d220-4d14-9374-ab47e05f5b6c" (UID: "aa05ff05-d220-4d14-9374-ab47e05f5b6c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:26:27.352715 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.352674    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-maas-api-tls\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:26:27.352715 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.352706    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpl2p\" (UniqueName: \"kubernetes.io/projected/aa05ff05-d220-4d14-9374-ab47e05f5b6c-kube-api-access-zpl2p\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:26:27.352715 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.352716    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa05ff05-d220-4d14-9374-ab47e05f5b6c-openshift-service-ca\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:26:27.352715 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.352726    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa05ff05-d220-4d14-9374-ab47e05f5b6c-metrics-certs\") on node \"ip-10-0-136-55.ec2.internal\" DevicePath \"\""
Sep 11 13:26:27.872994 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.872955    2564 generic.go:356] "Generic (PLEG): container finished" podID="aa05ff05-d220-4d14-9374-ab47e05f5b6c" containerID="aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b" exitCode=0
Sep 11 13:26:27.873420 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.873005    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"
Sep 11 13:26:27.873420 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.873044    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn" event={"ID":"aa05ff05-d220-4d14-9374-ab47e05f5b6c","Type":"ContainerDied","Data":"aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b"}
Sep 11 13:26:27.873420 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.873101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn" event={"ID":"aa05ff05-d220-4d14-9374-ab47e05f5b6c","Type":"ContainerDied","Data":"25e97d5923550514bbaebf4f359e4ab94fc109cd39b12fa97dddabab318559bf"}
Sep 11 13:26:27.873420 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.873127    2564 scope.go:117] "RemoveContainer" containerID="aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b"
Sep 11 13:26:27.883520 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.883502    2564 scope.go:117] "RemoveContainer" containerID="aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b"
Sep 11 13:26:27.883911 ip-10-0-136-55 kubenswrapper[2564]: E0911 13:26:27.883891    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b\": container with ID starting with aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b not found: ID does not exist" containerID="aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b"
Sep 11 13:26:27.883975 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.883921    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b"} err="failed to get container status \"aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b\": rpc error: code = NotFound desc = could not find container \"aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b\": container with ID starting with aa50061f6f01cff4c0eeb3b427a1790b948c17dae64a5d36100be53095f8106b not found: ID does not exist"
Sep 11 13:26:27.897463 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.897282    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"]
Sep 11 13:26:27.899218 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:27.899197    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7ff8fbcd4f-tjwrn"]
Sep 11 13:26:28.484663 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:28.484606    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa05ff05-d220-4d14-9374-ab47e05f5b6c" path="/var/lib/kubelet/pods/aa05ff05-d220-4d14-9374-ab47e05f5b6c/volumes"
Sep 11 13:26:57.917188 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:57.917155    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dksr2_bd3ef560-832e-431e-a746-46c3b241ab9a/manager/0.log"
Sep 11 13:26:58.048249 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:58.048214    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-758b8949b9-6njfb_a76cbc95-05b1-415d-a2c0-db0770311311/manager/0.log"
Sep 11 13:26:58.286077 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:58.285998    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r64c7_c45ed500-705c-46d0-87e4-e7e95a677b30/manager/0.log"
Sep 11 13:26:58.405063 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:58.405032    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cbb4999dc-gw4wv_814827b4-7a34-4e89-a771-d9cf1df08c6c/copy-manifests/0.log"
Sep 11 13:26:58.411795 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:58.411771    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cbb4999dc-gw4wv_814827b4-7a34-4e89-a771-d9cf1df08c6c/manager/0.log"
Sep 11 13:26:59.157010 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:59.156982    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7f866d8d77-86g5w_647074c8-18b1-4285-8d02-5781a3c80e6f/maas-api/0.log"
Sep 11 13:26:59.408215 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:26:59.408133    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-jpqmg_38ec5f72-e3b3-4e64-a818-57fc316658b4/postgres/0.log"
Sep 11 13:27:00.503546 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:00.503519    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-qc7zd_874f2689-6569-45ab-9536-60929b5bcaa6/manager/0.log"
Sep 11 13:27:00.745457 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:00.745419    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-cngsb_71158af1-0f29-445f-a7e4-6565e6dd1bc7/manager/0.log"
Sep 11 13:27:01.525152 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:01.525117    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gf682_f63e06ae-e731-41f0-a0a9-8b5e82ac5a26/discovery/0.log"
Sep 11 13:27:01.644062 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:01.644029    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75775448dd-chzl2_e22da6bd-847e-483d-9e31-7cfe9771817a/kube-auth-proxy/0.log"
Sep 11 13:27:02.017459 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:02.017423    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rhvp5_5bfac1c8-4103-480f-aad9-afa5be62df45/payload-pre-processing/0.log"
Sep 11 13:27:02.139437 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:02.139408    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dsj2b_82825f9e-f8bd-4fcc-b301-eaa5031c3f6e/payload-processing/0.log"
Sep 11 13:27:02.991819 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:02.991788    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pc5w5_f883abaa-2ee4-443d-be64-0cdb2bb7c58f/main/0.log"
Sep 11 13:27:03.111391 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:03.111358    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7pv7n_523a084f-cd1d-4297-ae38-0cee6ca3696a/main/0.log"
Sep 11 13:27:12.073524 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:12.073487    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2btsm_b32090af-80e4-4ac6-b9b2-da62336a920f/global-pull-secret-syncer/0.log"
Sep 11 13:27:12.241100 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:12.241070    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9j6hc_498c45c8-775f-460c-a2f6-18d1e5dfe451/konnectivity-agent/0.log"
Sep 11 13:27:12.355061 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:12.355028    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-55.ec2.internal_f0dbe8941313665e9751104dfb30bc72/haproxy/0.log"
Sep 11 13:27:12.654984 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:12.654904    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:27:12.659587 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:12.659563    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dscjx_8ad4f65c-09c8-4c72-969f-4ac3834c855d/ovn-acl-logging/0.log"
Sep 11 13:27:16.975308 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:16.975274    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-qc7zd_874f2689-6569-45ab-9536-60929b5bcaa6/manager/0.log"
Sep 11 13:27:17.078818 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:17.078786    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-cngsb_71158af1-0f29-445f-a7e4-6565e6dd1bc7/manager/0.log"
Sep 11 13:27:19.149139 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:19.148715    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j7g44_eb35d9cb-a077-48dd-bff8-8416801e46b0/node-exporter/0.log"
Sep 11 13:27:19.174147 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:19.174113    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j7g44_eb35d9cb-a077-48dd-bff8-8416801e46b0/kube-rbac-proxy/0.log"
Sep 11 13:27:19.201845 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:19.201812    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j7g44_eb35d9cb-a077-48dd-bff8-8416801e46b0/init-textfile/0.log"
Sep 11 13:27:19.433534 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:19.433510    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-lqtlx_68694e75-9dc4-41a8-9a0d-6b8b07d638c9/kube-rbac-proxy-main/0.log"
Sep 11 13:27:19.473729 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:19.473697    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-lqtlx_68694e75-9dc4-41a8-9a0d-6b8b07d638c9/kube-rbac-proxy-self/0.log"
Sep 11 13:27:19.505092 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:19.505066    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-lqtlx_68694e75-9dc4-41a8-9a0d-6b8b07d638c9/openshift-state-metrics/0.log"
Sep 11 13:27:19.823394 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:19.823322    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-mk5hl_85beb836-f149-472f-ae2a-449d9224c542/prometheus-operator-admission-webhook/0.log"
Sep 11 13:27:19.959401 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:19.959373    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9f49dc4d-zdr8h_6f351282-6bb3-49b0-98a7-3629e2d4a3aa/thanos-query/0.log"
Sep 11 13:27:19.985592 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:19.985564    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9f49dc4d-zdr8h_6f351282-6bb3-49b0-98a7-3629e2d4a3aa/kube-rbac-proxy-web/0.log"
Sep 11 13:27:20.013188 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:20.013156    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9f49dc4d-zdr8h_6f351282-6bb3-49b0-98a7-3629e2d4a3aa/kube-rbac-proxy/0.log"
Sep 11 13:27:20.039597 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:20.039566    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9f49dc4d-zdr8h_6f351282-6bb3-49b0-98a7-3629e2d4a3aa/prom-label-proxy/0.log"
Sep 11 13:27:20.069688 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:20.069660    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9f49dc4d-zdr8h_6f351282-6bb3-49b0-98a7-3629e2d4a3aa/kube-rbac-proxy-rules/0.log"
Sep 11 13:27:20.096340 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:20.096312    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9f49dc4d-zdr8h_6f351282-6bb3-49b0-98a7-3629e2d4a3aa/kube-rbac-proxy-metrics/0.log"
Sep 11 13:27:22.072865 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.072830    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"]
Sep 11 13:27:22.073269 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.073256    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa05ff05-d220-4d14-9374-ab47e05f5b6c" containerName="maas-api"
Sep 11 13:27:22.073314 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.073271    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa05ff05-d220-4d14-9374-ab47e05f5b6c" containerName="maas-api"
Sep 11 13:27:22.073385 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.073374    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa05ff05-d220-4d14-9374-ab47e05f5b6c" containerName="maas-api"
Sep 11 13:27:22.076598 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.076573    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.079840 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.079816    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vrzcr\"/\"openshift-service-ca.crt\""
Sep 11 13:27:22.080758 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.080740    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vrzcr\"/\"kube-root-ca.crt\""
Sep 11 13:27:22.080867 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.080745    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vrzcr\"/\"default-dockercfg-fwnzv\""
Sep 11 13:27:22.086715 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.086687    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"]
Sep 11 13:27:22.177156 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.177109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-sys\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.177156 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.177160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p2n8\" (UniqueName: \"kubernetes.io/projected/1381143a-9ea0-4c70-9727-ef53949254d3-kube-api-access-9p2n8\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.177419 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.177190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-podres\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.177419 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.177271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-lib-modules\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.177419 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.177299    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-proc\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.278805 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.278763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-lib-modules\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.279013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.278819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-proc\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.279013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.278881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-sys\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.279013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.278905    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p2n8\" (UniqueName: \"kubernetes.io/projected/1381143a-9ea0-4c70-9727-ef53949254d3-kube-api-access-9p2n8\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.279013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.278932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-podres\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.279013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.278965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-sys\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.279013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.278965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-proc\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.279013 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.278966    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-lib-modules\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.279317 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.279125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1381143a-9ea0-4c70-9727-ef53949254d3-podres\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.288136 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.288103    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p2n8\" (UniqueName: \"kubernetes.io/projected/1381143a-9ea0-4c70-9727-ef53949254d3-kube-api-access-9p2n8\") pod \"perf-node-gather-daemonset-lcfg7\" (UID: \"1381143a-9ea0-4c70-9727-ef53949254d3\") " pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.388214 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.388175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:22.663383 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:22.663356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"]
Sep 11 13:27:22.665233 ip-10-0-136-55 kubenswrapper[2564]: W0911 13:27:22.665205    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1381143a_9ea0_4c70_9727_ef53949254d3.slice/crio-ba8bce8cc3d89cc4deedbcb16fae23ef6c39fc0fc94ceeea1b4b478261d01540 WatchSource:0}: Error finding container ba8bce8cc3d89cc4deedbcb16fae23ef6c39fc0fc94ceeea1b4b478261d01540: Status 404 returned error can't find the container with id ba8bce8cc3d89cc4deedbcb16fae23ef6c39fc0fc94ceeea1b4b478261d01540
Sep 11 13:27:23.177057 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:23.177021    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7" event={"ID":"1381143a-9ea0-4c70-9727-ef53949254d3","Type":"ContainerStarted","Data":"0f11c16c89c8b4967415a6f1e79b8f3b0d67875b75c67e6efab476a98dae78c7"}
Sep 11 13:27:23.177486 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:23.177083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7" event={"ID":"1381143a-9ea0-4c70-9727-ef53949254d3","Type":"ContainerStarted","Data":"ba8bce8cc3d89cc4deedbcb16fae23ef6c39fc0fc94ceeea1b4b478261d01540"}
Sep 11 13:27:23.177486 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:23.177115    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:23.194135 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:23.194066    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7" podStartSLOduration=1.194047485 podStartE2EDuration="1.194047485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:27:23.193586372 +0000 UTC m=+2411.255893332" watchObservedRunningTime="2026-09-11 13:27:23.194047485 +0000 UTC m=+2411.256354444"
Sep 11 13:27:24.734348 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:24.734322    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rt827_4b04ebea-779c-4b76-9ebb-d66ffa34ee02/dns/0.log"
Sep 11 13:27:24.758991 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:24.758961    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rt827_4b04ebea-779c-4b76-9ebb-d66ffa34ee02/kube-rbac-proxy/0.log"
Sep 11 13:27:24.841177 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:24.841147    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7lzcr_439fc464-12e7-415c-922c-c2ed1971e596/dns-node-resolver/0.log"
Sep 11 13:27:25.473765 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:25.473703    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7d89946d9d-54mhv_f8c10a83-cc71-43f8-b38a-476e15357fa8/registry/0.log"
Sep 11 13:27:25.547445 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:25.547409    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zbccg_5d9fee76-abbc-4b67-bfaf-3fe94c1e6353/node-ca/0.log"
Sep 11 13:27:26.869692 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:26.869662    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gf682_f63e06ae-e731-41f0-a0a9-8b5e82ac5a26/discovery/0.log"
Sep 11 13:27:26.891347 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:26.891302    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75775448dd-chzl2_e22da6bd-847e-483d-9e31-7cfe9771817a/kube-auth-proxy/0.log"
Sep 11 13:27:26.997491 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:26.997458    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rhvp5_5bfac1c8-4103-480f-aad9-afa5be62df45/payload-pre-processing/0.log"
Sep 11 13:27:27.032152 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:27.032101    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dsj2b_82825f9e-f8bd-4fcc-b301-eaa5031c3f6e/payload-processing/0.log"
Sep 11 13:27:27.999876 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:27.999815    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nlctv_53e588b7-21cd-4c00-bfb7-68002b45b7a0/serve-healthcheck-canary/0.log"
Sep 11 13:27:28.531282 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:28.531244    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-qzskx_cdad3f72-dc2b-423e-8cd5-cbb2445667fb/insights-operator/0.log"
Sep 11 13:27:28.531700 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:28.531682    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-qzskx_cdad3f72-dc2b-423e-8cd5-cbb2445667fb/insights-operator/1.log"
Sep 11 13:27:28.559105 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:28.559081    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2c55d_b8a891b1-0df9-4a78-9936-82eea74ff958/kube-rbac-proxy/0.log"
Sep 11 13:27:28.580694 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:28.580671    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2c55d_b8a891b1-0df9-4a78-9936-82eea74ff958/exporter/0.log"
Sep 11 13:27:28.604309 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:28.604282    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2c55d_b8a891b1-0df9-4a78-9936-82eea74ff958/extractor/0.log"
Sep 11 13:27:29.192957 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:29.192930    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vrzcr/perf-node-gather-daemonset-lcfg7"
Sep 11 13:27:31.503334 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:31.503294    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dksr2_bd3ef560-832e-431e-a746-46c3b241ab9a/manager/0.log"
Sep 11 13:27:31.557796 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:31.557758    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-758b8949b9-6njfb_a76cbc95-05b1-415d-a2c0-db0770311311/manager/0.log"
Sep 11 13:27:31.665039 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:31.664958    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r64c7_c45ed500-705c-46d0-87e4-e7e95a677b30/manager/0.log"
Sep 11 13:27:31.697855 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:31.697819    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cbb4999dc-gw4wv_814827b4-7a34-4e89-a771-d9cf1df08c6c/manager/0.log"
Sep 11 13:27:31.734284 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:31.734256    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cbb4999dc-gw4wv_814827b4-7a34-4e89-a771-d9cf1df08c6c/copy-manifests/0.log"
Sep 11 13:27:33.350300 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:33.350272    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gf682_f63e06ae-e731-41f0-a0a9-8b5e82ac5a26/discovery/0.log"
Sep 11 13:27:33.366787 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:33.366755    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75775448dd-chzl2_e22da6bd-847e-483d-9e31-7cfe9771817a/kube-auth-proxy/0.log"
Sep 11 13:27:33.447319 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:33.447290    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rhvp5_5bfac1c8-4103-480f-aad9-afa5be62df45/payload-pre-processing/0.log"
Sep 11 13:27:33.475539 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:33.475510    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dsj2b_82825f9e-f8bd-4fcc-b301-eaa5031c3f6e/payload-processing/0.log"
Sep 11 13:27:35.222983 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:35.222952    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gf682_f63e06ae-e731-41f0-a0a9-8b5e82ac5a26/discovery/0.log"
Sep 11 13:27:35.237785 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:35.237756    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75775448dd-chzl2_e22da6bd-847e-483d-9e31-7cfe9771817a/kube-auth-proxy/0.log"
Sep 11 13:27:35.328960 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:35.328924    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-rhvp5_5bfac1c8-4103-480f-aad9-afa5be62df45/payload-pre-processing/0.log"
Sep 11 13:27:35.360827 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:35.360792    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dsj2b_82825f9e-f8bd-4fcc-b301-eaa5031c3f6e/payload-processing/0.log"
Sep 11 13:27:38.684235 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:38.684147    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-fsbsq_1f93c707-6182-4d68-a7e0-4e6260ff41c5/migrator/0.log"
Sep 11 13:27:38.704721 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:38.704693    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-fsbsq_1f93c707-6182-4d68-a7e0-4e6260ff41c5/graceful-termination/0.log"
Sep 11 13:27:39.093872 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:39.093841    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-l2k5w_c859ee38-4f38-46b2-8b8b-ae6345444bc6/kube-storage-version-migrator-operator/1.log"
Sep 11 13:27:39.094608 ip-10-0-136-55 kubenswrapper[2564]: I0911 13:27:39.094589    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-l2k5w_c859ee38-4f38-46b2-8b8b-ae6345444bc6/kube-storage-version-migrator-operator/0.log"