Sep 11 17:38:52.009424 ip-10-0-140-215 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 17:38:52.390057 ip-10-0-140-215 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 17:38:52.390057 ip-10-0-140-215 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 17:38:52.390057 ip-10-0-140-215 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 17:38:52.390057 ip-10-0-140-215 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 17:38:52.390057 ip-10-0-140-215 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 17:38:52.392232 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.392102    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 17:38:52.398773 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398755    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:52.398773 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398772    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:52.398773 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398776    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398780    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398783    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398786    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398788    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398793    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398796    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398799    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398801    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398804    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398807    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398810    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398814    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398817    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398819    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398822    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398825    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398827    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398830    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398832    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:52.398865 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398835    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398838    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398840    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398843    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398845    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398848    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398850    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398853    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398855    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398858    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398861    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398863    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398866    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398868    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398871    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398874    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398877    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398879    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398882    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398885    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:52.399360 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398894    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398898    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398903    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398907    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398909    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398912    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398914    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398917    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398919    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398922    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398924    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398927    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398929    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398932    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398935    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398938    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398940    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398943    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398946    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:52.399863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398949    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398951    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398954    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398956    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398960    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398963    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398966    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398968    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398971    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398973    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398978    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398981    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398986    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398989    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398991    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398994    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398997    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.398999    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399002    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399005    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:52.400339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399009    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399011    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399014    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399021    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399025    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399029    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399033    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399036    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399039    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399042    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399044    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399047    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399050    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399052    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399056    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399058    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399061    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399063    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399788    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:52.400814 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399796    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399799    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399802    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399804    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399807    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399809    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399812    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399815    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399818    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399820    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399823    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399826    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399828    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399831    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399833    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399836    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399838    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399841    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399845    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399849    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:52.401304 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399852    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399855    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399859    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399862    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399865    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399868    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399871    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399874    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399877    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399879    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399882    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399884    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399887    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399890    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399892    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399895    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399897    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399900    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399902    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:52.401845 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399905    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399908    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399910    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399913    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399915    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399918    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399920    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399923    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399925    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399928    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399930    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399933    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399936    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399940    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399943    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399945    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399948    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399951    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399953    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399956    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:52.402332 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399958    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399961    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399963    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399965    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399968    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399971    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399973    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399975    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399978    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399980    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399983    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399985    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399988    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399990    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399993    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399996    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.399998    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400001    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400003    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400007    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:52.402810 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400009    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400012    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400014    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400017    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400019    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400022    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400025    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400029    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400032    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400034    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400037    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400041    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400044    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400047    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400049    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400052    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400054    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400057    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400059    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:52.403340 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400441    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400446    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400449    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400452    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400454    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400457    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400461    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400464    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400466    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400469    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400471    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400474    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400476    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400479    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400481    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400483    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400486    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400489    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400492    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400494    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:52.403898 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400497    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400499    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400502    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400504    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400507    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400510    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400512    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400516    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400518    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400540    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400543    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400547    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400550    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400553    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400556    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400559    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400561    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400564    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400567    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400570    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400573    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:52.404399 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400576    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400578    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400581    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400583    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400586    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400588    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400591    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400593    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400596    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400599    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400601    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400604    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400607    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400610    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400614    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400617    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400620    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400622    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400625    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:52.404924 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400627    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400630    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400634    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400637    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400640    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400643    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400646    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400648    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400650    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400653    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400655    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400658    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400665    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400667    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400670    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400672    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400675    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400677    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400680    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:52.405401 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400683    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400685    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400688    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400691    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400693    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400696    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400699    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400702    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400704    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400707    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400709    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400711    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400715    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400719    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400722    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400724    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400727    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400730    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400733    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:52.405869 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.400735    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400806    2582 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400839    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400846    2582 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400850    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400854    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400858    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400862    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400867    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400873    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400876    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400880    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400883    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400886    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400890    2582 flags.go:64] FLAG: --cgroup-root=""
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400893    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400896    2582 flags.go:64] FLAG: --client-ca-file=""
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400899    2582 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400902    2582 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400906    2582 flags.go:64] FLAG: --cluster-domain=""
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400909    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400911    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400915    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400918    2582 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 17:38:52.406337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400921    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400925    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400928    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400931    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400934    2582 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400936    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400939    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400942    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400946    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400950    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400953    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400955    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400958    2582 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400961    2582 flags.go:64] FLAG: --enable-server="true"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400964    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400969    2582 flags.go:64] FLAG: --event-burst="100"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400979    2582 flags.go:64] FLAG: --event-qps="50"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400984    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400987    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400990    2582 flags.go:64] FLAG: --eviction-hard=""
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400994    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.400997    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401000    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401004    2582 flags.go:64] FLAG: --eviction-soft=""
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401007    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 17:38:52.406937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401010    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401013    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401016    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401019    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401022    2582 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401024    2582 flags.go:64] FLAG: --feature-gates=""
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401028    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401031    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401034    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401037    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401040    2582 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401043    2582 flags.go:64] FLAG: --help="false"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401046    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401049    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401052    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401054    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401058    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401061    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401064    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401067    2582 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401069    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401072    2582 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401075    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401093    2582 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 17:38:52.407593 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401098    2582 flags.go:64] FLAG: --kube-reserved=""
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401101    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401104    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401107    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401110    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401113    2582 flags.go:64] FLAG: --lock-file=""
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401117    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401120    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401123    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401129    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401131    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401134    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401137    2582 flags.go:64] FLAG: --logging-format="text"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401140    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401143    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401146    2582 flags.go:64] FLAG: --manifest-url=""
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401150    2582 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401154    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401156    2582 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401163    2582 flags.go:64] FLAG: --max-pods="110"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401166    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401169    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401172    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401175    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401178    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 17:38:52.408187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401181    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401184    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401191    2582 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401193    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401196    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401199    2582 flags.go:64] FLAG: --pod-cidr=""
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401202    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401208    2582 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401218    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401221    2582 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401225    2582 flags.go:64] FLAG: --port="10250"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401228    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401231    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06f0b629be0363804"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401235    2582 flags.go:64] FLAG: --qos-reserved=""
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401238    2582 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401241    2582 flags.go:64] FLAG: --register-node="true"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401244    2582 flags.go:64] FLAG: --register-with-taints=""
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401248    2582 flags.go:64] FLAG: --registry-burst="10"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401250    2582 flags.go:64] FLAG: --registry-qps="5"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401253    2582 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401256    2582 flags.go:64] FLAG: --reserved-memory=""
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401260    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401263    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401266    2582 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401269    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 17:38:52.408798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401271    2582 flags.go:64] FLAG: --runonce="false"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401274    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401277    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401280    2582 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401283    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401286    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401288    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401291    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401294    2582 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401297    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401300    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401303    2582 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401305    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401308    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401311    2582 flags.go:64] FLAG: --system-cgroups=""
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401315    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401329    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401332    2582 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401335    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401856    2582 flags.go:64] FLAG: --tls-min-version=""
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401861    2582 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401864    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401868    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401871    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401874    2582 flags.go:64] FLAG: --v="2"
Sep 11 17:38:52.409423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401878    2582 flags.go:64] FLAG: --version="false"
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401882    2582 flags.go:64] FLAG: --vmodule=""
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401886    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.401889    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402032    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402035    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402038    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402041    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402044    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402047    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402049    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402053    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402057    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402060    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402063    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402065    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402068    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402071    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402073    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402089    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:52.410027 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402092    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402095    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402098    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402101    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402109    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402112    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402114    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402118    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402121    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402123    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402126    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402129    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402131    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402134    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402136    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402139    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402141    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402144    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402147    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402149    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:52.410863 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402152    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402155    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402157    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402159    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402162    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402165    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402168    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402172    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402175    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402178    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402181    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402184    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402186    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402188    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402191    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402194    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402196    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402204    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402207    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:52.411667 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402211    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402214    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402216    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402219    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402221    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402224    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402226    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402228    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402231    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402233    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402236    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402238    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402241    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402243    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402246    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402248    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402251    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402253    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402257    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402260    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:52.412181 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402262    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402265    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402267    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402270    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402272    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402274    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402277    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402279    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402282    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402284    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402287    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402295    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402299    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402301    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402304    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402306    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402308    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402311    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402314    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402316    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:52.412672 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402319    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:52.413171 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402321    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:52.413171 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402324    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:52.413171 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.402332    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:52.413171 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.403031    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 17:38:52.413171 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.412734    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 17:38:52.413171 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.412752    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 17:38:52.413716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.413704    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 17:38:52.413746 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.413716    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 17:38:52.413786 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413777    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:52.413786 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413785    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413789    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413792    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413795    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413797    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413800    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413803    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413806    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413808    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413811    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413813    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413816    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413819    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413822    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413825    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413827    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413830    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413833    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413835    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413838    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:52.413841 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413842    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413844    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413847    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413850    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413853    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413855    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413858    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413861    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413864    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413867    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413869    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413872    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413875    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413877    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413880    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413882    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413887    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413891    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413894    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413897    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:52.414347 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413900    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413903    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413905    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413908    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413911    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413921    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413924    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413927    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413929    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413932    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413934    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413936    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413939    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413941    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413945    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413948    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413951    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413953    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413956    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413959    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:52.414830 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413961    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413965    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413968    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413971    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413974    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413977    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413979    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413982    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413984    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413988    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413992    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413995    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.413997    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414000    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414002    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414005    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414007    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414010    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414013    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:52.415339 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414016    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414018    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414021    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414023    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414026    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414028    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414031    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414034    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414037    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414040    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414042    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414045    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414047    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414050    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414052    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414055    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414057    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414060    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:52.415816 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414062    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.414067    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414198    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414203    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414205    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414208    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414211    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414214    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414218    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414221    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414223    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414226    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414229    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414232    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414236    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:52.416256 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414239    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414241    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414244    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414247    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414249    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414252    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414255    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414258    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414261    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414265    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414269    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414272    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414276    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414279    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414281    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414284    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414287    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414289    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414292    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414294    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:52.416633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414297    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414299    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414302    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414304    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414307    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414309    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414312    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414314    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414316    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414319    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414321    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414324    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414327    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414330    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414332    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414335    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414337    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414340    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414343    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414346    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:52.417157 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414348    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414351    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414354    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414357    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414359    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414363    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414366    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414368    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414371    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414373    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414376    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414378    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414381    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414383    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414386    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414388    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414391    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414393    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414395    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414398    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:52.417640 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414400    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414403    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414405    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414407    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414410    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414413    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414415    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414418    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414421    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414424    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414428    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414431    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414434    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414437    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414440    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414443    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414446    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414448    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414451    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:52.418185 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414454    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:52.418642 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414456    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:52.418642 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414459    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:52.418642 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414462    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:52.418642 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414464    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:52.418642 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414467    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:52.418642 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:52.414469    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:52.418642 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.414474    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 17:38:52.418642 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.414608    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 17:38:52.419035 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.419021    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 17:38:52.420177 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.420140    2582 server.go:1019] "Starting client certificate rotation"
Sep 11 17:38:52.420278 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.420255    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 17:38:52.420334 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.420323    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 17:38:52.423780 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.423765    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 17:38:52.425012 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.424978    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 17:38:52.438861 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.438832    2582 log.go:25] "Validated CRI v1 runtime API"
Sep 11 17:38:52.442271 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.442256    2582 log.go:25] "Validated CRI v1 image API"
Sep 11 17:38:52.443124 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.443109    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 17:38:52.446505 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.446477    2582 fs.go:135] Filesystem UUIDs: map[754eddd5-9b9c-4c07-b7fc-084480af55c3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cdfbbfe2-1855-4a55-9f2f-f6e431e40915:/dev/nvme0n1p3]
Sep 11 17:38:52.446598 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.446504    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 17:38:52.448844 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.448824    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 17:38:52.453690 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.453579    2582 manager.go:216] Machine: {Timestamp:2026-09-11 17:38:52.452148101 +0000 UTC m=+0.340326438 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101051 MemoryCapacity:32812146688 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26c7c435d7e2ce04ae86f74fde9bc2 SystemUUID:ec26c7c4-35d7-e2ce-04ae-86f74fde9bc2 BootID:b149b8d5-8b53-4e5b-8734-3f87dfea0fff Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406073344 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b8:27:97:9c:af Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b8:27:97:9c:af Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:53:fc:92:3f:b9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812146688 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 17:38:52.453690 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.453680    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 17:38:52.453808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.453765    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 17:38:52.457229 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.457196    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 17:38:52.457373 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.457233    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-215.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 17:38:52.458731 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.458722    2582 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 17:38:52.458769 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.458734    2582 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 17:38:52.458769 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.458742    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 17:38:52.459566 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.459556    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 17:38:52.460858 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.460849    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 17:38:52.465159 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.465147    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 17:38:52.465278 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.465268    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 17:38:52.465313 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.465300    2582 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 17:38:52.465313 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.465310    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 17:38:52.465363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.465328    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 17:38:52.465363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.465337    2582 kubelet.go:398] "Adding apiserver pod source"
Sep 11 17:38:52.465363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.465346    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 17:38:52.465759 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.465747    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 17:38:52.465808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.465766    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 17:38:52.468334 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.468317    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 17:38:52.470729 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.470714    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 17:38:52.470787 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.470745    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 17:38:52.472679 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472664    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 17:38:52.472741 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472687    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 17:38:52.472741 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472697    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 17:38:52.472741 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472706    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 17:38:52.472741 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472718    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 17:38:52.472741 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472728    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 17:38:52.472878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472747    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 17:38:52.472878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472755    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 17:38:52.472878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472764    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 17:38:52.472878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472772    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 17:38:52.472878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472784    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 17:38:52.472878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.472793    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 17:38:52.473762 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.473747    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 17:38:52.473762 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.473762    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 17:38:52.475684 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.475517    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 17:38:52.475750 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.475530    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 17:38:52.477626 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.477612    2582 server.go:1268] "Started kubelet"
Sep 11 17:38:52.477724 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.477699    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 17:38:52.477774 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.477752    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 17:38:52.477865 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.477854    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 17:38:52.478473 ip-10-0-140-215 systemd[1]: Started Kubernetes Kubelet.
Sep 11 17:38:52.480100 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.480068    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 17:38:52.480791 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.480776    2582 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 17:38:52.483583 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.483563    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 17:38:52.484882 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.484869    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-215.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 17:38:52.488294 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.488273    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 17:38:52.488360 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.485795    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 17:38:52.488418 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.484983    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-215.ec2.internal.18d454edd7cf06f9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-215.ec2.internal,UID:ip-10-0-140-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-215.ec2.internal,},FirstTimestamp:2026-09-11 17:38:52.477581049 +0000 UTC m=+0.365759386,LastTimestamp:2026-09-11 17:38:52.477581049 +0000 UTC m=+0.365759386,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-215.ec2.internal,}"
Sep 11 17:38:52.488562 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.488544    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-215.ec2.internal\" not found"
Sep 11 17:38:52.489101 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.489063    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 17:38:52.489189 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.489064    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 17:38:52.490455 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.489133    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 17:38:52.490531 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.490524    2582 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 17:38:52.490590 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.490534    2582 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 17:38:52.492106 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.492070    2582 factory.go:55] Registering systemd factory
Sep 11 17:38:52.492190 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.492128    2582 factory.go:223] Registration of the systemd container factory successfully
Sep 11 17:38:52.492386 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.492361    2582 factory.go:153] Registering CRI-O factory
Sep 11 17:38:52.492386 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.492389    2582 factory.go:223] Registration of the crio container factory successfully
Sep 11 17:38:52.492526 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.492456    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 17:38:52.492526 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.492474    2582 factory.go:103] Registering Raw factory
Sep 11 17:38:52.492526 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.492488    2582 manager.go:1196] Started watching for new ooms in manager
Sep 11 17:38:52.492961 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.492938    2582 manager.go:349] Starting recovery of all containers
Sep 11 17:38:52.493292 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.493269    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 17:38:52.493405 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.493327    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 17:38:52.502823 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.502807    2582 manager.go:354] Recovery completed
Sep 11 17:38:52.506580 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.506568    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:52.508379 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.508355    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:52.508452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.508397    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:52.508452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.508412    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:52.509014 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.509002    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 17:38:52.509014 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.509013    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 17:38:52.509102 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.509021    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:52.509102 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.509028    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 17:38:52.509696 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.509683    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:52.509754 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.509706    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:52.509754 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.509717    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:52.510822 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.510811    2582 policy_none.go:49] "None policy: Start"
Sep 11 17:38:52.510860 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.510827    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 17:38:52.510860 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.510842    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 17:38:52.511050 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.510984    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-215.ec2.internal.18d454edd9a4fb47  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-215.ec2.internal,UID:ip-10-0-140-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-215.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-215.ec2.internal,},FirstTimestamp:2026-09-11 17:38:52.508379975 +0000 UTC m=+0.396558313,LastTimestamp:2026-09-11 17:38:52.508379975 +0000 UTC m=+0.396558313,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-215.ec2.internal,}"
Sep 11 17:38:52.512483 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.512471    2582 policy_none.go:47] "Start"
Sep 11 17:38:52.526406 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.526339    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-215.ec2.internal.18d454edd9a5574f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-215.ec2.internal,UID:ip-10-0-140-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-215.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-215.ec2.internal,},FirstTimestamp:2026-09-11 17:38:52.508403535 +0000 UTC m=+0.396581873,LastTimestamp:2026-09-11 17:38:52.508403535 +0000 UTC m=+0.396581873,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-215.ec2.internal,}"
Sep 11 17:38:52.538657 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.538576    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-215.ec2.internal.18d454edd9a58bad  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-215.ec2.internal,UID:ip-10-0-140-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-215.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-215.ec2.internal,},FirstTimestamp:2026-09-11 17:38:52.508416941 +0000 UTC m=+0.396595278,LastTimestamp:2026-09-11 17:38:52.508416941 +0000 UTC m=+0.396595278,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-215.ec2.internal,}"
Sep 11 17:38:52.547700 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.547627    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-215.ec2.internal.18d454edd9a4fb47\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-215.ec2.internal.18d454edd9a4fb47  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-215.ec2.internal,UID:ip-10-0-140-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-215.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-215.ec2.internal,},FirstTimestamp:2026-09-11 17:38:52.508379975 +0000 UTC m=+0.396558313,LastTimestamp:2026-09-11 17:38:52.509694862 +0000 UTC m=+0.397873199,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-215.ec2.internal,}"
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.556796    2582 manager.go:337] "Starting Device Plugin manager"
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.556831    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.556841    2582 server.go:87] "Starting device plugin registration server"
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.557247    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.557260    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.557448    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.557519    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.557527    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.558273    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.558330    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-215.ec2.internal\" not found"
Sep 11 17:38:52.571295 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.562221    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-215.ec2.internal.18d454edd9a5574f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-215.ec2.internal.18d454edd9a5574f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-215.ec2.internal,UID:ip-10-0-140-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-215.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-215.ec2.internal,},FirstTimestamp:2026-09-11 17:38:52.508403535 +0000 UTC m=+0.396581873,LastTimestamp:2026-09-11 17:38:52.509711119 +0000 UTC m=+0.397889456,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-215.ec2.internal,}"
Sep 11 17:38:52.572690 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.572663    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fj6zw"
Sep 11 17:38:52.574715 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.574649    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-215.ec2.internal.18d454edd9a58bad\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-215.ec2.internal.18d454edd9a58bad  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-215.ec2.internal,UID:ip-10-0-140-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-215.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-215.ec2.internal,},FirstTimestamp:2026-09-11 17:38:52.508416941 +0000 UTC m=+0.396595278,LastTimestamp:2026-09-11 17:38:52.509721458 +0000 UTC m=+0.397899796,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-215.ec2.internal,}"
Sep 11 17:38:52.581999 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.581933    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-215.ec2.internal.18d454eddcacffc9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-215.ec2.internal,UID:ip-10-0-140-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-140-215.ec2.internal,},FirstTimestamp:2026-09-11 17:38:52.559237065 +0000 UTC m=+0.447415392,LastTimestamp:2026-09-11 17:38:52.559237065 +0000 UTC m=+0.447415392,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-215.ec2.internal,}"
Sep 11 17:38:52.588402 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.588385    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fj6zw"
Sep 11 17:38:52.625794 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.625755    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 17:38:52.627203 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.627176    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 17:38:52.627295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.627241    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 17:38:52.627295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.627269    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 17:38:52.627377 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.627317    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 17:38:52.641299 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.641219    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:52.657789 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.657760    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:52.658632 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.658615    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:52.658706 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.658646    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:52.658706 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.658658    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:52.658706 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.658680    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.675884 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.675866    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.675931 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.675887    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-215.ec2.internal\": node \"ip-10-0-140-215.ec2.internal\" not found"
Sep 11 17:38:52.705442 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.705410    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-215.ec2.internal\" not found"
Sep 11 17:38:52.728106 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.728063    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal"]
Sep 11 17:38:52.728180 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.728150    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:52.729618 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.729602    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:52.729686 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.729630    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:52.729686 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.729642    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:52.731856 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.731845    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:52.731972 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.731955    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.732013 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.731993    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:52.732618 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.732601    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:52.732716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.732630    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:52.732716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.732639    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:52.732716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.732607    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:52.732716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.732710    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:52.732860 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.732721    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:52.735025 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.735009    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.735122 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.735040    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:52.735782 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.735764    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:52.735868 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.735795    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:52.735868 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.735805    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:52.758661 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.758643    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-215.ec2.internal\" not found" node="ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.762998 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.762978    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-215.ec2.internal\" not found" node="ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.793310 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.793281    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1961536c6add67b3e31d5e4dec17d1b8-config\") pod \"kube-apiserver-proxy-ip-10-0-140-215.ec2.internal\" (UID: \"1961536c6add67b3e31d5e4dec17d1b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.793310 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.793306    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/95d8349af1ea27705737de7491da6301-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal\" (UID: \"95d8349af1ea27705737de7491da6301\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.793479 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.793325    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95d8349af1ea27705737de7491da6301-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal\" (UID: \"95d8349af1ea27705737de7491da6301\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.805921 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.805895    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-215.ec2.internal\" not found"
Sep 11 17:38:52.894295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.894215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1961536c6add67b3e31d5e4dec17d1b8-config\") pod \"kube-apiserver-proxy-ip-10-0-140-215.ec2.internal\" (UID: \"1961536c6add67b3e31d5e4dec17d1b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.894295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.894245    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/95d8349af1ea27705737de7491da6301-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal\" (UID: \"95d8349af1ea27705737de7491da6301\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.894295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.894263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95d8349af1ea27705737de7491da6301-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal\" (UID: \"95d8349af1ea27705737de7491da6301\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.894493 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.894327    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95d8349af1ea27705737de7491da6301-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal\" (UID: \"95d8349af1ea27705737de7491da6301\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.894493 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.894351    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1961536c6add67b3e31d5e4dec17d1b8-config\") pod \"kube-apiserver-proxy-ip-10-0-140-215.ec2.internal\" (UID: \"1961536c6add67b3e31d5e4dec17d1b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.894493 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:52.894389    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/95d8349af1ea27705737de7491da6301-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal\" (UID: \"95d8349af1ea27705737de7491da6301\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:52.906143 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:52.906119    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-215.ec2.internal\" not found"
Sep 11 17:38:53.007012 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.006974    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-215.ec2.internal\" not found"
Sep 11 17:38:53.061212 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.061175    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:53.066586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.066296    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:53.107516 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.107485    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-215.ec2.internal\" not found"
Sep 11 17:38:53.208093 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.207993    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-215.ec2.internal\" not found"
Sep 11 17:38:53.308549 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.308511    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-215.ec2.internal\" not found"
Sep 11 17:38:53.335852 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.335823    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:53.389200 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.389170    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:53.400264 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.400236    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 17:38:53.402058 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.402038    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal"
Sep 11 17:38:53.412737 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.412718    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 17:38:53.419871 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.419854    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 17:38:53.419978 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.419955    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 17:38:53.420015 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.420003    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 17:38:53.466225 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.466164    2582 apiserver.go:52] "Watching apiserver"
Sep 11 17:38:53.483023 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.482996    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 17:38:53.484091 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.484051    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal","openshift-image-registry/node-ca-wkt44","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal","openshift-network-diagnostics/network-check-target-2bq7k","openshift-network-operator/iptables-alerter-8rs4d","openshift-ovn-kubernetes/ovnkube-node-k8kz5","kube-system/global-pull-secret-syncer-cvgh7","kube-system/konnectivity-agent-qccvc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs","openshift-cluster-node-tuning-operator/tuned-9tf5b","openshift-dns/node-resolver-88zgk","openshift-multus/multus-additional-cni-plugins-6wm7m","openshift-multus/multus-k4gcm","openshift-multus/network-metrics-daemon-l458g"]
Sep 11 17:38:53.487075 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.487054    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:53.487181 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.487159    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:38:53.488628 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.488611    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 17:38:53.489638 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.489621    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.492615 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.492570    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:53.492615 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.492584    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 17:38:53.492615 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.492612    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tbbxd\""
Sep 11 17:38:53.492815 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.492659    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:38:53.492815 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.492586    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 17:38:53.492815 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.492589    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 17:38:53.492977 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.492944    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 17:38:53.495741 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.495721    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:38:53.495847 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.495780    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:38:53.498105 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498050    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-var-lib-cni-bin\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498216 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-daemon-config\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498216 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498136    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:53.498216 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498160    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46c761dc-4a0b-419e-8c89-86f14b88b4a9-kubelet-config\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:53.498216 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498183    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-cni-dir\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498446 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498225    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/820309c1-3f9d-4ec8-8117-e39b368a4a50-cni-binary-copy\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498446 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498247    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-run-k8s-cni-cncf-io\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498446 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498298    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:53.498446 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-system-cni-dir\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498446 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498341    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-cnibin\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498446 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-var-lib-kubelet\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498446 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498418    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-conf-dir\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498464    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.498810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498442    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-run-multus-certs\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498512    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46c761dc-4a0b-419e-8c89-86f14b88b4a9-dbus\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:53.498810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498535    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-run-netns\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498559    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-var-lib-cni-multus\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498582    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-hostroot\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498604    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-etc-kubernetes\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498626    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg4wv\" (UniqueName: \"kubernetes.io/projected/820309c1-3f9d-4ec8-8117-e39b368a4a50-kube-api-access-kg4wv\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.498810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498668    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkj6w\" (UniqueName: \"kubernetes.io/projected/1cde1024-eadf-4e40-8354-b153794c1109-kube-api-access-zkj6w\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:53.498810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.498693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-socket-dir-parent\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.501157 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.501138    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.501684 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.501665    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 17:38:53.501754 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.501702    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:38:53.501754 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.501733    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8nnz8\""
Sep 11 17:38:53.501754 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.501739    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 17:38:53.501965 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.501952    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 17:38:53.503820 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.503805    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.505945 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.505924    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 17:38:53.506031 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.505959    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 17:38:53.506031 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.505982    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xqr5d\""
Sep 11 17:38:53.506314 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.506299    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 17:38:53.506350 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.506339    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 17:38:53.506928 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.506913    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:38:53.507614 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.507597    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 17:38:53.507762 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.507746    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 17:38:53.507831 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.507792    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 17:38:53.507892 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.507826    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 17:38:53.507892 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.507857    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-48mhc\""
Sep 11 17:38:53.508037 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.508022    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 17:38:53.509324 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.509305    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jk9gf\""
Sep 11 17:38:53.509449 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.509389    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 17:38:53.509509 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.509391    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 17:38:53.509562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.509553    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.512056 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.512039    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.512238 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.512220    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 17:38:53.512939 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.512921    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 17:38:53.513022 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.512921    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 17:38:53.513073 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.513033    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lnz8n\""
Sep 11 17:38:53.514153 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.514138    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.516330 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.516313    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.519023 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.519007    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:38:53.519121 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.519038    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 17:38:53.519673 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.519660    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tf97q\""
Sep 11 17:38:53.520216 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.520204    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vkpkh\""
Sep 11 17:38:53.521145 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.521133    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 17:38:53.521318 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.521307    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 17:38:53.522198 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.522180    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-glnbx\""
Sep 11 17:38:53.523428 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.523407    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 17:38:53.525632 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.525614    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 17:38:53.541426 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.541403    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v6th7"
Sep 11 17:38:53.553230 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.553213    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v6th7"
Sep 11 17:38:53.559354 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.559333    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:53.589897 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.589879    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 17:38:53.593350 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.593313    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 17:33:53 +0000 UTC" deadline="2028-06-08 08:26:48.800805022 +0000 UTC"
Sep 11 17:38:53.593401 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.593352    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15254h47m55.207456706s"
Sep 11 17:38:53.598905 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.598871    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx6m9\" (UniqueName: \"kubernetes.io/projected/1ac45d7c-f7d4-422d-8e43-3cbc853fe1df-kube-api-access-fx6m9\") pod \"node-resolver-88zgk\" (UID: \"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df\") " pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.598984 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.598923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-cni-dir\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.598984 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.598950    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-run-systemd\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.599063 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.598986    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-run-openvswitch\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.599063 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599010    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.599063 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599035    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-run\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.599063 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599042    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-cni-dir\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.599063 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599058    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-lib-modules\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.599287 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599124    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-conf-dir\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.599287 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599166    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.599287 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599188    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snnrh\" (UniqueName: \"kubernetes.io/projected/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-kube-api-access-snnrh\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.599287 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599186    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-conf-dir\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.599287 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599203    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-modprobe-d\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.599287 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599239    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ac45d7c-f7d4-422d-8e43-3cbc853fe1df-hosts-file\") pod \"node-resolver-88zgk\" (UID: \"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df\") " pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.599287 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599261    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-run-netns\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.599287 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-run-ovn\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599294    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bqt5\" (UniqueName: \"kubernetes.io/projected/015e5b25-5f9b-4c49-b6ac-575f4065c9ea-kube-api-access-6bqt5\") pod \"node-ca-wkt44\" (UID: \"015e5b25-5f9b-4c49-b6ac-575f4065c9ea\") " pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599306    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-run-netns\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599312    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599367    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-slash\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.599386    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599402    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-ovnkube-config\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.599432    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs podName:1cde1024-eadf-4e40-8354-b153794c1109 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:54.099417653 +0000 UTC m=+1.987595997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs") pod "network-metrics-daemon-l458g" (UID: "1cde1024-eadf-4e40-8354-b153794c1109") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599444    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/015e5b25-5f9b-4c49-b6ac-575f4065c9ea-serviceca\") pod \"node-ca-wkt44\" (UID: \"015e5b25-5f9b-4c49-b6ac-575f4065c9ea\") " pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599457    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-sys\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599481    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac45d7c-f7d4-422d-8e43-3cbc853fe1df-tmp-dir\") pod \"node-resolver-88zgk\" (UID: \"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df\") " pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599501    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/265ef6f1-4e31-464b-81b6-5ba76a7510bc-system-cni-dir\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599520    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-device-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599546    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46c761dc-4a0b-419e-8c89-86f14b88b4a9-kubelet-config\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599578    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-kubelet\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599604    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-etc-openvswitch\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.599635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599638    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46c761dc-4a0b-419e-8c89-86f14b88b4a9-kubelet-config\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599681    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-node-log\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599706    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-log-socket\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599728    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-cni-bin\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599751    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1238055a-bea2-4089-8439-1d0fc275c735-konnectivity-ca\") pod \"konnectivity-agent-qccvc\" (UID: \"1238055a-bea2-4089-8439-1d0fc275c735\") " pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg4wv\" (UniqueName: \"kubernetes.io/projected/820309c1-3f9d-4ec8-8117-e39b368a4a50-kube-api-access-kg4wv\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-system-cni-dir\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-cnibin\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599879    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-var-lib-kubelet\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599893    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-ovn-node-metrics-cert\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-kubernetes\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599947    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-var-lib-kubelet\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599945    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46c761dc-4a0b-419e-8c89-86f14b88b4a9-dbus\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599957    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-cnibin\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599973    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-hostroot\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.599997    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-system-cni-dir\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600005    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-env-overrides\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600024    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46c761dc-4a0b-419e-8c89-86f14b88b4a9-dbus\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:53.600266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600058    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-hostroot\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600059    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xktwr\" (UniqueName: \"kubernetes.io/projected/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-kube-api-access-xktwr\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600106    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-sysctl-conf\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600139    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-systemd\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600179    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-var-lib-kubelet\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600206    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-tuned\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-socket-dir-parent\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600255    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-var-lib-cni-bin\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600275    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-var-lib-openvswitch\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600288    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-socket-dir-parent\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600297    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-sys-fs\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600317    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32fd6194-d269-4529-a491-ac6e45a0874e-host-slash\") pod \"iptables-alerter-8rs4d\" (UID: \"32fd6194-d269-4529-a491-ac6e45a0874e\") " pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600329    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-var-lib-cni-bin\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600335    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkmbz\" (UniqueName: \"kubernetes.io/projected/32fd6194-d269-4529-a491-ac6e45a0874e-kube-api-access-bkmbz\") pod \"iptables-alerter-8rs4d\" (UID: \"32fd6194-d269-4529-a491-ac6e45a0874e\") " pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600362    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dfc2f18-39da-47dd-8933-e9c9cd04231a-tmp\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600385    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/265ef6f1-4e31-464b-81b6-5ba76a7510bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/820309c1-3f9d-4ec8-8117-e39b368a4a50-cni-binary-copy\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600513    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-run-k8s-cni-cncf-io\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600552    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/265ef6f1-4e31-464b-81b6-5ba76a7510bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600581    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc4ht\" (UniqueName: \"kubernetes.io/projected/265ef6f1-4e31-464b-81b6-5ba76a7510bc-kube-api-access-fc4ht\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600605    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-systemd-units\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600606    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-run-k8s-cni-cncf-io\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600664    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-run-multus-certs\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600679    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-ovnkube-script-lib\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600718    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-run-multus-certs\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600737    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-etc-selinux\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.600742    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-var-lib-cni-multus\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.600802    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret podName:46c761dc-4a0b-419e-8c89-86f14b88b4a9 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:54.100786494 +0000 UTC m=+1.988964823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret") pod "global-pull-secret-syncer-cvgh7" (UID: "46c761dc-4a0b-419e-8c89-86f14b88b4a9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600821    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/820309c1-3f9d-4ec8-8117-e39b368a4a50-cni-binary-copy\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600843    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkj6w\" (UniqueName: \"kubernetes.io/projected/1cde1024-eadf-4e40-8354-b153794c1109-kube-api-access-zkj6w\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600819    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-host-var-lib-cni-multus\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.601732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600882    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-daemon-config\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-cni-netd\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600932    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/265ef6f1-4e31-464b-81b6-5ba76a7510bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600955    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-run-netns\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-registration-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.600999    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-sysctl-d\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601024    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1238055a-bea2-4089-8439-1d0fc275c735-agent-certs\") pod \"konnectivity-agent-qccvc\" (UID: \"1238055a-bea2-4089-8439-1d0fc275c735\") " pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601049    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-etc-kubernetes\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601072    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-socket-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601105    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/820309c1-3f9d-4ec8-8117-e39b368a4a50-etc-kubernetes\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601115    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32fd6194-d269-4529-a491-ac6e45a0874e-iptables-alerter-script\") pod \"iptables-alerter-8rs4d\" (UID: \"32fd6194-d269-4529-a491-ac6e45a0874e\") " pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601140    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-sysconfig\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601161    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/265ef6f1-4e31-464b-81b6-5ba76a7510bc-cnibin\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601206    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pftgq\" (UniqueName: \"kubernetes.io/projected/0dfc2f18-39da-47dd-8933-e9c9cd04231a-kube-api-access-pftgq\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601238    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/265ef6f1-4e31-464b-81b6-5ba76a7510bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601261    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z4jn\" (UniqueName: \"kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn\") pod \"network-check-target-2bq7k\" (UID: \"1580cdd2-f034-4b8a-a78d-5b41d050d18a\") " pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:38:53.602222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601286    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-run-ovn-kubernetes\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.602667 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601305    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/015e5b25-5f9b-4c49-b6ac-575f4065c9ea-host\") pod \"node-ca-wkt44\" (UID: \"015e5b25-5f9b-4c49-b6ac-575f4065c9ea\") " pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.602667 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601326    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-host\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.602667 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.601365    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/820309c1-3f9d-4ec8-8117-e39b368a4a50-multus-daemon-config\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.606468 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.606450    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 17:38:53.610028 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.610003    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkj6w\" (UniqueName: \"kubernetes.io/projected/1cde1024-eadf-4e40-8354-b153794c1109-kube-api-access-zkj6w\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:53.610123 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.610003    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg4wv\" (UniqueName: \"kubernetes.io/projected/820309c1-3f9d-4ec8-8117-e39b368a4a50-kube-api-access-kg4wv\") pod \"multus-k4gcm\" (UID: \"820309c1-3f9d-4ec8-8117-e39b368a4a50\") " pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.676582 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.676400    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d8349af1ea27705737de7491da6301.slice/crio-b6b58e5cbca6023b4cedcc3cae06bcab98e6f7503b5ea75d09fec665e8b0479e WatchSource:0}: Error finding container b6b58e5cbca6023b4cedcc3cae06bcab98e6f7503b5ea75d09fec665e8b0479e: Status 404 returned error can't find the container with id b6b58e5cbca6023b4cedcc3cae06bcab98e6f7503b5ea75d09fec665e8b0479e
Sep 11 17:38:53.677018 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.676990    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1961536c6add67b3e31d5e4dec17d1b8.slice/crio-35ffc34043eb762a47a44cb7860145be29b82f32e464de57eb27b8f9d30c11e9 WatchSource:0}: Error finding container 35ffc34043eb762a47a44cb7860145be29b82f32e464de57eb27b8f9d30c11e9: Status 404 returned error can't find the container with id 35ffc34043eb762a47a44cb7860145be29b82f32e464de57eb27b8f9d30c11e9
Sep 11 17:38:53.681186 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.681171    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:38:53.702190 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702163    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-kubelet\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702190 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702192    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-etc-openvswitch\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702208    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-node-log\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702230    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-log-socket\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702253    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-cni-bin\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702276    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1238055a-bea2-4089-8439-1d0fc275c735-konnectivity-ca\") pod \"konnectivity-agent-qccvc\" (UID: \"1238055a-bea2-4089-8439-1d0fc275c735\") " pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:38:53.702358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702285    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-kubelet\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702297    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-node-log\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702316    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-log-socket\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702325    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-cni-bin\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702284    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-etc-openvswitch\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702354    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-ovn-node-metrics-cert\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702377    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-kubernetes\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702403    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-env-overrides\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702423    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-kubernetes\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702427    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xktwr\" (UniqueName: \"kubernetes.io/projected/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-kube-api-access-xktwr\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702455    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-sysctl-conf\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702479    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-systemd\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-var-lib-kubelet\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-tuned\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702543    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-var-lib-openvswitch\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702564    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-sys-fs\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702586    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32fd6194-d269-4529-a491-ac6e45a0874e-host-slash\") pod \"iptables-alerter-8rs4d\" (UID: \"32fd6194-d269-4529-a491-ac6e45a0874e\") " pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702600    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-var-lib-kubelet\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702607    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkmbz\" (UniqueName: \"kubernetes.io/projected/32fd6194-d269-4529-a491-ac6e45a0874e-kube-api-access-bkmbz\") pod \"iptables-alerter-8rs4d\" (UID: \"32fd6194-d269-4529-a491-ac6e45a0874e\") " pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702604    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-sysctl-conf\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702649    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-systemd\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702659    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32fd6194-d269-4529-a491-ac6e45a0874e-host-slash\") pod \"iptables-alerter-8rs4d\" (UID: \"32fd6194-d269-4529-a491-ac6e45a0874e\") " pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.702757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702637    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dfc2f18-39da-47dd-8933-e9c9cd04231a-tmp\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702699    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-var-lib-openvswitch\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702705    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/265ef6f1-4e31-464b-81b6-5ba76a7510bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702744    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/265ef6f1-4e31-464b-81b6-5ba76a7510bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702772    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc4ht\" (UniqueName: \"kubernetes.io/projected/265ef6f1-4e31-464b-81b6-5ba76a7510bc-kube-api-access-fc4ht\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702807    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-systemd-units\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702844    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-ovnkube-script-lib\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702869    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-etc-selinux\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702902    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-cni-netd\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702940    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/265ef6f1-4e31-464b-81b6-5ba76a7510bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702965    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-run-netns\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-registration-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703017    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-env-overrides\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703039    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-sysctl-d\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703063    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1238055a-bea2-4089-8439-1d0fc275c735-agent-certs\") pod \"konnectivity-agent-qccvc\" (UID: \"1238055a-bea2-4089-8439-1d0fc275c735\") " pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-socket-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703133    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1238055a-bea2-4089-8439-1d0fc275c735-konnectivity-ca\") pod \"konnectivity-agent-qccvc\" (UID: \"1238055a-bea2-4089-8439-1d0fc275c735\") " pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:38:53.703809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703143    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-etc-selinux\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703136    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32fd6194-d269-4529-a491-ac6e45a0874e-iptables-alerter-script\") pod \"iptables-alerter-8rs4d\" (UID: \"32fd6194-d269-4529-a491-ac6e45a0874e\") " pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-sysconfig\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/265ef6f1-4e31-464b-81b6-5ba76a7510bc-cnibin\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703220    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-systemd-units\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703237    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pftgq\" (UniqueName: \"kubernetes.io/projected/0dfc2f18-39da-47dd-8933-e9c9cd04231a-kube-api-access-pftgq\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703265    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/265ef6f1-4e31-464b-81b6-5ba76a7510bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4jn\" (UniqueName: \"kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn\") pod \"network-check-target-2bq7k\" (UID: \"1580cdd2-f034-4b8a-a78d-5b41d050d18a\") " pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703350    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-run-ovn-kubernetes\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/015e5b25-5f9b-4c49-b6ac-575f4065c9ea-host\") pod \"node-ca-wkt44\" (UID: \"015e5b25-5f9b-4c49-b6ac-575f4065c9ea\") " pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703395    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-host\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx6m9\" (UniqueName: \"kubernetes.io/projected/1ac45d7c-f7d4-422d-8e43-3cbc853fe1df-kube-api-access-fx6m9\") pod \"node-resolver-88zgk\" (UID: \"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df\") " pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703446    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-run-systemd\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703469    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-run-openvswitch\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703474    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/265ef6f1-4e31-464b-81b6-5ba76a7510bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703495    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703518    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-run\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.704597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703540    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-lib-modules\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703586    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/265ef6f1-4e31-464b-81b6-5ba76a7510bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703602    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snnrh\" (UniqueName: \"kubernetes.io/projected/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-kube-api-access-snnrh\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-modprobe-d\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703684    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ac45d7c-f7d4-422d-8e43-3cbc853fe1df-hosts-file\") pod \"node-resolver-88zgk\" (UID: \"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df\") " pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703701    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-ovnkube-script-lib\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703713    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-run-ovn\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.702846    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-sys-fs\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703728    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-sysconfig\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703747    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bqt5\" (UniqueName: \"kubernetes.io/projected/015e5b25-5f9b-4c49-b6ac-575f4065c9ea-kube-api-access-6bqt5\") pod \"node-ca-wkt44\" (UID: \"015e5b25-5f9b-4c49-b6ac-575f4065c9ea\") " pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703774    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/265ef6f1-4e31-464b-81b6-5ba76a7510bc-cnibin\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703185    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-cni-netd\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-slash\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703814    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-ovnkube-config\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/015e5b25-5f9b-4c49-b6ac-575f4065c9ea-serviceca\") pod \"node-ca-wkt44\" (UID: \"015e5b25-5f9b-4c49-b6ac-575f4065c9ea\") " pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703860    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-sys\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.705400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703883    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac45d7c-f7d4-422d-8e43-3cbc853fe1df-tmp-dir\") pod \"node-resolver-88zgk\" (UID: \"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df\") " pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703914    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/265ef6f1-4e31-464b-81b6-5ba76a7510bc-system-cni-dir\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.703946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-device-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704039    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/265ef6f1-4e31-464b-81b6-5ba76a7510bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704060    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32fd6194-d269-4529-a491-ac6e45a0874e-iptables-alerter-script\") pod \"iptables-alerter-8rs4d\" (UID: \"32fd6194-d269-4529-a491-ac6e45a0874e\") " pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704127    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-run-netns\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704155    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-modprobe-d\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704175    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-slash\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704180    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-registration-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704183    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-run-ovn-kubernetes\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704202    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ac45d7c-f7d4-422d-8e43-3cbc853fe1df-hosts-file\") pod \"node-resolver-88zgk\" (UID: \"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df\") " pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704043    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-device-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704218    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/015e5b25-5f9b-4c49-b6ac-575f4065c9ea-host\") pod \"node-ca-wkt44\" (UID: \"015e5b25-5f9b-4c49-b6ac-575f4065c9ea\") " pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704250    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-run-openvswitch\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704264    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704298    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-socket-dir\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704130    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-run-ovn\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.705950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704367    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-run\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704449    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-lib-modules\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704457    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-host\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704488    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-run-systemd\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704495    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704549    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-sys\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704633    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-sysctl-d\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704649    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-ovnkube-config\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704676    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/265ef6f1-4e31-464b-81b6-5ba76a7510bc-system-cni-dir\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704723    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac45d7c-f7d4-422d-8e43-3cbc853fe1df-tmp-dir\") pod \"node-resolver-88zgk\" (UID: \"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df\") " pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.704882    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/015e5b25-5f9b-4c49-b6ac-575f4065c9ea-serviceca\") pod \"node-ca-wkt44\" (UID: \"015e5b25-5f9b-4c49-b6ac-575f4065c9ea\") " pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.705098    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/265ef6f1-4e31-464b-81b6-5ba76a7510bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.705550    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0dfc2f18-39da-47dd-8933-e9c9cd04231a-etc-tuned\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.705720    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-ovn-node-metrics-cert\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.706452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.706002    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dfc2f18-39da-47dd-8933-e9c9cd04231a-tmp\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.706853 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.706838    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1238055a-bea2-4089-8439-1d0fc275c735-agent-certs\") pod \"konnectivity-agent-qccvc\" (UID: \"1238055a-bea2-4089-8439-1d0fc275c735\") " pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:38:53.711757 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.711733    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:38:53.711839 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.711762    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:38:53.711839 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.711776    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4z4jn for pod openshift-network-diagnostics/network-check-target-2bq7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:53.711839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.711828    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkmbz\" (UniqueName: \"kubernetes.io/projected/32fd6194-d269-4529-a491-ac6e45a0874e-kube-api-access-bkmbz\") pod \"iptables-alerter-8rs4d\" (UID: \"32fd6194-d269-4529-a491-ac6e45a0874e\") " pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.711949 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:53.711863    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn podName:1580cdd2-f034-4b8a-a78d-5b41d050d18a nodeName:}" failed. No retries permitted until 2026-09-11 17:38:54.211845402 +0000 UTC m=+2.100023729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4z4jn" (UniqueName: "kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn") pod "network-check-target-2bq7k" (UID: "1580cdd2-f034-4b8a-a78d-5b41d050d18a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:53.713102 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.713068    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xktwr\" (UniqueName: \"kubernetes.io/projected/7ad358f4-54dc-4aea-9e7f-3387e4f91b7c-kube-api-access-xktwr\") pod \"ovnkube-node-k8kz5\" (UID: \"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.713231 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.713216    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pftgq\" (UniqueName: \"kubernetes.io/projected/0dfc2f18-39da-47dd-8933-e9c9cd04231a-kube-api-access-pftgq\") pod \"tuned-9tf5b\" (UID: \"0dfc2f18-39da-47dd-8933-e9c9cd04231a\") " pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.717175 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.717123    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc4ht\" (UniqueName: \"kubernetes.io/projected/265ef6f1-4e31-464b-81b6-5ba76a7510bc-kube-api-access-fc4ht\") pod \"multus-additional-cni-plugins-6wm7m\" (UID: \"265ef6f1-4e31-464b-81b6-5ba76a7510bc\") " pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.719947 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.719922    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bqt5\" (UniqueName: \"kubernetes.io/projected/015e5b25-5f9b-4c49-b6ac-575f4065c9ea-kube-api-access-6bqt5\") pod \"node-ca-wkt44\" (UID: \"015e5b25-5f9b-4c49-b6ac-575f4065c9ea\") " pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.720166 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.720148    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snnrh\" (UniqueName: \"kubernetes.io/projected/0db99c1e-299d-4572-8b5f-fbe7c21e2e73-kube-api-access-snnrh\") pod \"aws-ebs-csi-driver-node-dfgxs\" (UID: \"0db99c1e-299d-4572-8b5f-fbe7c21e2e73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.721211 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.721195    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx6m9\" (UniqueName: \"kubernetes.io/projected/1ac45d7c-f7d4-422d-8e43-3cbc853fe1df-kube-api-access-fx6m9\") pod \"node-resolver-88zgk\" (UID: \"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df\") " pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.804965 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.804936    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:53.814453 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.814431    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k4gcm"
Sep 11 17:38:53.824649 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.824626    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod820309c1_3f9d_4ec8_8117_e39b368a4a50.slice/crio-eb05bec0ce49ba463c68cef83da0700bc34d93684b0d24ca3839baeef3014179 WatchSource:0}: Error finding container eb05bec0ce49ba463c68cef83da0700bc34d93684b0d24ca3839baeef3014179: Status 404 returned error can't find the container with id eb05bec0ce49ba463c68cef83da0700bc34d93684b0d24ca3839baeef3014179
Sep 11 17:38:53.847604 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.847569    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8rs4d"
Sep 11 17:38:53.856284 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.856262    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:38:53.856543 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.856517    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32fd6194_d269_4529_a491_ac6e45a0874e.slice/crio-c6694321c8055038d103d2818b14bd3b887b01ed8b9ab94f7baefb7fdb184e30 WatchSource:0}: Error finding container c6694321c8055038d103d2818b14bd3b887b01ed8b9ab94f7baefb7fdb184e30: Status 404 returned error can't find the container with id c6694321c8055038d103d2818b14bd3b887b01ed8b9ab94f7baefb7fdb184e30
Sep 11 17:38:53.862178 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.862154    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs"
Sep 11 17:38:53.865102 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.865031    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad358f4_54dc_4aea_9e7f_3387e4f91b7c.slice/crio-900f132fdad54d2d773f2f44196ff44a6e8f1f6a53c62fdcf4f3e57289db4ca5 WatchSource:0}: Error finding container 900f132fdad54d2d773f2f44196ff44a6e8f1f6a53c62fdcf4f3e57289db4ca5: Status 404 returned error can't find the container with id 900f132fdad54d2d773f2f44196ff44a6e8f1f6a53c62fdcf4f3e57289db4ca5
Sep 11 17:38:53.872583 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.872558    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db99c1e_299d_4572_8b5f_fbe7c21e2e73.slice/crio-e2b7a9b8eaf9bff2d37d08904ae0bd866973d6415f1f24998a4a468468b10094 WatchSource:0}: Error finding container e2b7a9b8eaf9bff2d37d08904ae0bd866973d6415f1f24998a4a468468b10094: Status 404 returned error can't find the container with id e2b7a9b8eaf9bff2d37d08904ae0bd866973d6415f1f24998a4a468468b10094
Sep 11 17:38:53.885978 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.885953    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:38:53.895169 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.895144    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1238055a_bea2_4089_8439_1d0fc275c735.slice/crio-14fd1bfd122ab120ce69630069ea21fdd2781d7ce1bb70823187ab8fdd8521f2 WatchSource:0}: Error finding container 14fd1bfd122ab120ce69630069ea21fdd2781d7ce1bb70823187ab8fdd8521f2: Status 404 returned error can't find the container with id 14fd1bfd122ab120ce69630069ea21fdd2781d7ce1bb70823187ab8fdd8521f2
Sep 11 17:38:53.900108 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.900092    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkt44"
Sep 11 17:38:53.908472 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.908446    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod015e5b25_5f9b_4c49_b6ac_575f4065c9ea.slice/crio-9d14a1209d101588eed79d7c6b0460012da3eddd0db1e304f8d539dc43df60ca WatchSource:0}: Error finding container 9d14a1209d101588eed79d7c6b0460012da3eddd0db1e304f8d539dc43df60ca: Status 404 returned error can't find the container with id 9d14a1209d101588eed79d7c6b0460012da3eddd0db1e304f8d539dc43df60ca
Sep 11 17:38:53.917337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.917319    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9tf5b"
Sep 11 17:38:53.925785 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.925762    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dfc2f18_39da_47dd_8933_e9c9cd04231a.slice/crio-fcc8e9e57e753d1a33da92a6bb4287c1fd889b5d9bec010f050cd6cbc5bb373e WatchSource:0}: Error finding container fcc8e9e57e753d1a33da92a6bb4287c1fd889b5d9bec010f050cd6cbc5bb373e: Status 404 returned error can't find the container with id fcc8e9e57e753d1a33da92a6bb4287c1fd889b5d9bec010f050cd6cbc5bb373e
Sep 11 17:38:53.931734 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.931716    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-88zgk"
Sep 11 17:38:53.936364 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:53.936347    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6wm7m"
Sep 11 17:38:53.940838 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.940815    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac45d7c_f7d4_422d_8e43_3cbc853fe1df.slice/crio-41270f7c3be45ce2f6846d4d8a938f65b077dad7d8efecad4371fade44be51bc WatchSource:0}: Error finding container 41270f7c3be45ce2f6846d4d8a938f65b077dad7d8efecad4371fade44be51bc: Status 404 returned error can't find the container with id 41270f7c3be45ce2f6846d4d8a938f65b077dad7d8efecad4371fade44be51bc
Sep 11 17:38:53.945167 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:38:53.945146    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265ef6f1_4e31_464b_81b6_5ba76a7510bc.slice/crio-a114504a4eea81781c18ae262d3cb3f970462312b9fde724b4232d1177576cf6 WatchSource:0}: Error finding container a114504a4eea81781c18ae262d3cb3f970462312b9fde724b4232d1177576cf6: Status 404 returned error can't find the container with id a114504a4eea81781c18ae262d3cb3f970462312b9fde724b4232d1177576cf6
Sep 11 17:38:54.106524 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.106433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:54.106524 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.106484    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:54.106712 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:54.106595    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:54.106712 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:54.106598    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:54.106712 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:54.106652    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret podName:46c761dc-4a0b-419e-8c89-86f14b88b4a9 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:55.106637839 +0000 UTC m=+2.994816165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret") pod "global-pull-secret-syncer-cvgh7" (UID: "46c761dc-4a0b-419e-8c89-86f14b88b4a9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:54.106712 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:54.106666    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs podName:1cde1024-eadf-4e40-8354-b153794c1109 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:55.106659696 +0000 UTC m=+2.994838022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs") pod "network-metrics-daemon-l458g" (UID: "1cde1024-eadf-4e40-8354-b153794c1109") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:54.307735 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.307622    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4jn\" (UniqueName: \"kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn\") pod \"network-check-target-2bq7k\" (UID: \"1580cdd2-f034-4b8a-a78d-5b41d050d18a\") " pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:38:54.307916 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:54.307835    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:38:54.307916 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:54.307858    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:38:54.307916 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:54.307869    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4z4jn for pod openshift-network-diagnostics/network-check-target-2bq7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:54.308038 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:54.307934    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn podName:1580cdd2-f034-4b8a-a78d-5b41d050d18a nodeName:}" failed. No retries permitted until 2026-09-11 17:38:55.30791282 +0000 UTC m=+3.196091145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4z4jn" (UniqueName: "kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn") pod "network-check-target-2bq7k" (UID: "1580cdd2-f034-4b8a-a78d-5b41d050d18a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:54.409977 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.409348    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:54.556391 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.556323    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 17:33:54 +0000 UTC" deadline="2028-06-19 01:34:54.269260774 +0000 UTC"
Sep 11 17:38:54.556391 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.556389    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15511h55m59.712876121s"
Sep 11 17:38:54.630396 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.630364    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:54.630580 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.630404    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:38:54.630580 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:54.630501    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:38:54.630699 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:54.630633    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:38:54.665427 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.665083    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qccvc" event={"ID":"1238055a-bea2-4089-8439-1d0fc275c735","Type":"ContainerStarted","Data":"14fd1bfd122ab120ce69630069ea21fdd2781d7ce1bb70823187ab8fdd8521f2"}
Sep 11 17:38:54.679861 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.679814    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8rs4d" event={"ID":"32fd6194-d269-4529-a491-ac6e45a0874e","Type":"ContainerStarted","Data":"c6694321c8055038d103d2818b14bd3b887b01ed8b9ab94f7baefb7fdb184e30"}
Sep 11 17:38:54.693025 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.692985    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal" event={"ID":"95d8349af1ea27705737de7491da6301","Type":"ContainerStarted","Data":"b6b58e5cbca6023b4cedcc3cae06bcab98e6f7503b5ea75d09fec665e8b0479e"}
Sep 11 17:38:54.709558 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.705019    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal" event={"ID":"1961536c6add67b3e31d5e4dec17d1b8","Type":"ContainerStarted","Data":"35ffc34043eb762a47a44cb7860145be29b82f32e464de57eb27b8f9d30c11e9"}
Sep 11 17:38:54.715179 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.715122    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wm7m" event={"ID":"265ef6f1-4e31-464b-81b6-5ba76a7510bc","Type":"ContainerStarted","Data":"a114504a4eea81781c18ae262d3cb3f970462312b9fde724b4232d1177576cf6"}
Sep 11 17:38:54.724864 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.724829    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-88zgk" event={"ID":"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df","Type":"ContainerStarted","Data":"41270f7c3be45ce2f6846d4d8a938f65b077dad7d8efecad4371fade44be51bc"}
Sep 11 17:38:54.732089 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.732053    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkt44" event={"ID":"015e5b25-5f9b-4c49-b6ac-575f4065c9ea","Type":"ContainerStarted","Data":"9d14a1209d101588eed79d7c6b0460012da3eddd0db1e304f8d539dc43df60ca"}
Sep 11 17:38:54.749716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.749611    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs" event={"ID":"0db99c1e-299d-4572-8b5f-fbe7c21e2e73","Type":"ContainerStarted","Data":"e2b7a9b8eaf9bff2d37d08904ae0bd866973d6415f1f24998a4a468468b10094"}
Sep 11 17:38:54.765317 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.765280    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5" event={"ID":"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c","Type":"ContainerStarted","Data":"900f132fdad54d2d773f2f44196ff44a6e8f1f6a53c62fdcf4f3e57289db4ca5"}
Sep 11 17:38:54.781626 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.781586    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k4gcm" event={"ID":"820309c1-3f9d-4ec8-8117-e39b368a4a50","Type":"ContainerStarted","Data":"eb05bec0ce49ba463c68cef83da0700bc34d93684b0d24ca3839baeef3014179"}
Sep 11 17:38:54.800691 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:54.800644    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9tf5b" event={"ID":"0dfc2f18-39da-47dd-8933-e9c9cd04231a","Type":"ContainerStarted","Data":"fcc8e9e57e753d1a33da92a6bb4287c1fd889b5d9bec010f050cd6cbc5bb373e"}
Sep 11 17:38:55.114408 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:55.113982    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:55.114408 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:55.114047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:55.114408 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:55.114222    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:55.114408 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:55.114285    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret podName:46c761dc-4a0b-419e-8c89-86f14b88b4a9 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:57.114265064 +0000 UTC m=+5.002443399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret") pod "global-pull-secret-syncer-cvgh7" (UID: "46c761dc-4a0b-419e-8c89-86f14b88b4a9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:55.114751 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:55.114691    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:55.114751 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:55.114744    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs podName:1cde1024-eadf-4e40-8354-b153794c1109 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:57.114727678 +0000 UTC m=+5.002906010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs") pod "network-metrics-daemon-l458g" (UID: "1cde1024-eadf-4e40-8354-b153794c1109") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:55.315129 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:55.315090    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4jn\" (UniqueName: \"kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn\") pod \"network-check-target-2bq7k\" (UID: \"1580cdd2-f034-4b8a-a78d-5b41d050d18a\") " pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:38:55.315344 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:55.315272    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:38:55.315344 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:55.315298    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:38:55.315344 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:55.315312    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4z4jn for pod openshift-network-diagnostics/network-check-target-2bq7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:55.315505 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:55.315419    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn podName:1580cdd2-f034-4b8a-a78d-5b41d050d18a nodeName:}" failed. No retries permitted until 2026-09-11 17:38:57.315385411 +0000 UTC m=+5.203563739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4z4jn" (UniqueName: "kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn") pod "network-check-target-2bq7k" (UID: "1580cdd2-f034-4b8a-a78d-5b41d050d18a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:55.557436 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:55.557362    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 17:33:54 +0000 UTC" deadline="2028-05-09 10:26:30.364234711 +0000 UTC"
Sep 11 17:38:55.557436 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:55.557402    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14536h47m34.806837827s"
Sep 11 17:38:55.628132 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:55.627528    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:55.628132 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:55.627694    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:38:56.180675 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:56.179293    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:56.628615 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:56.628528    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:56.629227 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:56.628670    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:38:56.631348 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:56.631320    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:38:56.631467 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:56.631435    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:38:57.130606 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:57.130563    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:57.130794 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:57.130676    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:57.130855 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:57.130833    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:57.130919 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:57.130901    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret podName:46c761dc-4a0b-419e-8c89-86f14b88b4a9 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:01.130881964 +0000 UTC m=+9.019060293 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret") pod "global-pull-secret-syncer-cvgh7" (UID: "46c761dc-4a0b-419e-8c89-86f14b88b4a9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:57.131463 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:57.131441    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:57.131601 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:57.131498    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs podName:1cde1024-eadf-4e40-8354-b153794c1109 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:01.131481318 +0000 UTC m=+9.019659643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs") pod "network-metrics-daemon-l458g" (UID: "1cde1024-eadf-4e40-8354-b153794c1109") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:57.333097 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:57.333043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4jn\" (UniqueName: \"kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn\") pod \"network-check-target-2bq7k\" (UID: \"1580cdd2-f034-4b8a-a78d-5b41d050d18a\") " pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:38:57.333353 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:57.333329    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:38:57.333427 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:57.333363    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:38:57.333427 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:57.333380    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4z4jn for pod openshift-network-diagnostics/network-check-target-2bq7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:57.333551 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:57.333447    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn podName:1580cdd2-f034-4b8a-a78d-5b41d050d18a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:01.33342531 +0000 UTC m=+9.221603644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4z4jn" (UniqueName: "kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn") pod "network-check-target-2bq7k" (UID: "1580cdd2-f034-4b8a-a78d-5b41d050d18a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:57.627524 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:57.627475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:57.627691 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:57.627636    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:38:58.632599 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:58.632557    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:38:58.633074 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:58.632700    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:38:58.633074 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:58.632986    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:38:58.633250 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:58.633104    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:38:59.628424 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:38:59.628385    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:38:59.628625 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:38:59.628538    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:00.628601 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:00.628561    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:00.629110 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:00.628726    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:00.629110 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:00.628561    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:00.629110 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:00.629102    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:01.167166 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:01.167120    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:01.167353 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:01.167322    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:01.167487 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:01.167470    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:39:01.167556 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:01.167542    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret podName:46c761dc-4a0b-419e-8c89-86f14b88b4a9 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:09.16752297 +0000 UTC m=+17.055701302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret") pod "global-pull-secret-syncer-cvgh7" (UID: "46c761dc-4a0b-419e-8c89-86f14b88b4a9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:39:01.167965 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:01.167946    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:39:01.168053 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:01.168001    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs podName:1cde1024-eadf-4e40-8354-b153794c1109 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:09.167984695 +0000 UTC m=+17.056163022 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs") pod "network-metrics-daemon-l458g" (UID: "1cde1024-eadf-4e40-8354-b153794c1109") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:39:01.369910 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:01.369250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4jn\" (UniqueName: \"kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn\") pod \"network-check-target-2bq7k\" (UID: \"1580cdd2-f034-4b8a-a78d-5b41d050d18a\") " pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:01.369910 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:01.369436    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:39:01.369910 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:01.369453    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:39:01.369910 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:01.369466    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4z4jn for pod openshift-network-diagnostics/network-check-target-2bq7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:39:01.369910 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:01.369534    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn podName:1580cdd2-f034-4b8a-a78d-5b41d050d18a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:09.369512695 +0000 UTC m=+17.257691024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4z4jn" (UniqueName: "kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn") pod "network-check-target-2bq7k" (UID: "1580cdd2-f034-4b8a-a78d-5b41d050d18a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:39:01.628067 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:01.627989    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:01.628228 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:01.628139    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:02.629096 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:02.629036    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:02.629573 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:02.629176    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:02.629660 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:02.629642    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:02.629777 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:02.629744    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:03.627551 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:03.627513    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:03.627764 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:03.627661    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:04.628584 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:04.628546    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:04.628584 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:04.628569    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:04.629165 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:04.628688    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:04.629165 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:04.628800    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:05.628347 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:05.628310    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:05.628553 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:05.628456    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:06.627953 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:06.627908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:06.628482 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:06.628056    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:06.628482 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:06.628118    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:06.628482 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:06.628197    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:07.628309 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:07.628266    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:07.628785 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:07.628421    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:08.627901 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:08.627862    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:08.628110 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:08.627993    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:08.628110 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:08.628063    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:08.628294 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:08.628230    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:09.241357 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:09.241313    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:09.241814 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:09.241376    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:09.241814 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:09.241482    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:39:09.241814 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:09.241488    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:39:09.241814 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:09.241541    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs podName:1cde1024-eadf-4e40-8354-b153794c1109 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:25.24152521 +0000 UTC m=+33.129703534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs") pod "network-metrics-daemon-l458g" (UID: "1cde1024-eadf-4e40-8354-b153794c1109") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:39:09.241814 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:09.241561    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret podName:46c761dc-4a0b-419e-8c89-86f14b88b4a9 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:25.241550237 +0000 UTC m=+33.129728566 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret") pod "global-pull-secret-syncer-cvgh7" (UID: "46c761dc-4a0b-419e-8c89-86f14b88b4a9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:39:09.442962 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:09.442919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4jn\" (UniqueName: \"kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn\") pod \"network-check-target-2bq7k\" (UID: \"1580cdd2-f034-4b8a-a78d-5b41d050d18a\") " pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:09.443179 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:09.443104    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:39:09.443179 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:09.443131    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:39:09.443179 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:09.443145    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4z4jn for pod openshift-network-diagnostics/network-check-target-2bq7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:39:09.443309 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:09.443199    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn podName:1580cdd2-f034-4b8a-a78d-5b41d050d18a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:25.443183896 +0000 UTC m=+33.331362220 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4z4jn" (UniqueName: "kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn") pod "network-check-target-2bq7k" (UID: "1580cdd2-f034-4b8a-a78d-5b41d050d18a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:39:09.628318 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:09.628244    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:09.628513 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:09.628381    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:10.628199 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:10.628160    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:10.628575 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:10.628208    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:10.628575 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:10.628278    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:10.628575 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:10.628388    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:11.627751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:11.627718    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:11.627941 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:11.627860    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:12.629098 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:12.629051    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:12.629503 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:12.629197    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:12.629503 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:12.629245    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:12.629503 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:12.629335    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:13.627441 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:13.627412    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:13.627609 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:13.627535    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:14.628340 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.627838    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:14.628340 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:14.628296    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:14.629444 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.629423    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:14.629552 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:14.629513    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:14.842400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.842370    2582 generic.go:356] "Generic (PLEG): container finished" podID="95d8349af1ea27705737de7491da6301" containerID="810d4f805e8ae19cfc5fd5a869f75200eae70fbf00655a0df6ff668bf9558f17" exitCode=0
Sep 11 17:39:14.842519 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.842442    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal" event={"ID":"95d8349af1ea27705737de7491da6301","Type":"ContainerDied","Data":"810d4f805e8ae19cfc5fd5a869f75200eae70fbf00655a0df6ff668bf9558f17"}
Sep 11 17:39:14.843954 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.843931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal" event={"ID":"1961536c6add67b3e31d5e4dec17d1b8","Type":"ContainerStarted","Data":"050a8c0f4b5e7b2af58e0d7231d42f266b7278b27b8c07b4a08d98e0ed29319f"}
Sep 11 17:39:14.845518 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.845492    2582 generic.go:356] "Generic (PLEG): container finished" podID="265ef6f1-4e31-464b-81b6-5ba76a7510bc" containerID="a7a1acf79d3d7418dcec70fad7eb8c19268215ab5687910af8f10baecbc449a0" exitCode=0
Sep 11 17:39:14.845614 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.845561    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wm7m" event={"ID":"265ef6f1-4e31-464b-81b6-5ba76a7510bc","Type":"ContainerDied","Data":"a7a1acf79d3d7418dcec70fad7eb8c19268215ab5687910af8f10baecbc449a0"}
Sep 11 17:39:14.847105 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.847068    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-88zgk" event={"ID":"1ac45d7c-f7d4-422d-8e43-3cbc853fe1df","Type":"ContainerStarted","Data":"49f694dcd1a111c38702fda6c91fb85e8b707584a97f8831c54564a133131d4c"}
Sep 11 17:39:14.848726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.848707    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkt44" event={"ID":"015e5b25-5f9b-4c49-b6ac-575f4065c9ea","Type":"ContainerStarted","Data":"f5d3d540b84de95621142ffa1dd148787a286c4123a85477f39e6ad1bdde8c39"}
Sep 11 17:39:14.850221 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.850179    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs" event={"ID":"0db99c1e-299d-4572-8b5f-fbe7c21e2e73","Type":"ContainerStarted","Data":"8949fd030870e23d33570c1ea32aa0b9692ae2bf7e33a48c39cc3fde139fe249"}
Sep 11 17:39:14.853816 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.853792    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5" event={"ID":"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c","Type":"ContainerStarted","Data":"351e9176cb8f8fdb43b958f2bc274114d729e99cd8b1f54e706e7da5bbce9b42"}
Sep 11 17:39:14.853942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.853858    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5" event={"ID":"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c","Type":"ContainerStarted","Data":"978698662de1fc637072f76b22c8176499f4b7e5b4762bfc8be5f3b40a55d7cb"}
Sep 11 17:39:14.853942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.853897    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5" event={"ID":"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c","Type":"ContainerStarted","Data":"650943d7792acb8605a401ff22a84a9a0412c497c10532b4ad9593a4b745a90d"}
Sep 11 17:39:14.853942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.853931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5" event={"ID":"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c","Type":"ContainerStarted","Data":"77706aeac66d4ec28d9e84fb76857d43b1b568364792999d1331d346e1a061c6"}
Sep 11 17:39:14.854111 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.853965    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5" event={"ID":"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c","Type":"ContainerStarted","Data":"dc6edd57bc6b6baef3669bf85ba1196697b71f4974bbc9ac48668613df92256b"}
Sep 11 17:39:14.855278 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.855260    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k4gcm" event={"ID":"820309c1-3f9d-4ec8-8117-e39b368a4a50","Type":"ContainerStarted","Data":"275c178d6f2f3f3146f9f11a24eaf56c696d0d06ef1f92aaf1b59fdd39f558b9"}
Sep 11 17:39:14.857755 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.857725    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9tf5b" event={"ID":"0dfc2f18-39da-47dd-8933-e9c9cd04231a","Type":"ContainerStarted","Data":"d249b9c76c6a26d39b84aca0e70f143288f3273bdbb99e8b63ddc3f1ca9a1b29"}
Sep 11 17:39:14.860162 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.860134    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qccvc" event={"ID":"1238055a-bea2-4089-8439-1d0fc275c735","Type":"ContainerStarted","Data":"44d0e58295094862a0564dd2d156c9fcfc3a49b622edd38ec3f2d04f5f38fd69"}
Sep 11 17:39:14.875455 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.875413    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wkt44" podStartSLOduration=1.641710231 podStartE2EDuration="21.875399294s" totalImagesPullingTime="20.233689063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:14.875111669 +0000 UTC m=+22.763290016" watchObservedRunningTime="2026-09-11 17:39:14.875399294 +0000 UTC m=+22.763577640"
Sep 11 17:39:14.889224 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.889174    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-88zgk" podStartSLOduration=1.6826113550000001 podStartE2EDuration="21.889159591s" totalImagesPullingTime="20.206548236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:14.888981456 +0000 UTC m=+22.777159824" watchObservedRunningTime="2026-09-11 17:39:14.889159591 +0000 UTC m=+22.777337953"
Sep 11 17:39:14.925732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.925685    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-215.ec2.internal" podStartSLOduration=20.925669748 podStartE2EDuration="20.925669748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:14.925152419 +0000 UTC m=+22.813330766" watchObservedRunningTime="2026-09-11 17:39:14.925669748 +0000 UTC m=+22.813848094"
Sep 11 17:39:14.943548 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.943500    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qccvc" podStartSLOduration=1.698129327 podStartE2EDuration="21.943484697s" totalImagesPullingTime="20.24535537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:14.942873385 +0000 UTC m=+22.831051733" watchObservedRunningTime="2026-09-11 17:39:14.943484697 +0000 UTC m=+22.831663037"
Sep 11 17:39:14.959480 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.959427    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9tf5b" podStartSLOduration=1.718426979 podStartE2EDuration="21.959411088s" totalImagesPullingTime="20.240984109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:14.959015184 +0000 UTC m=+22.847193524" watchObservedRunningTime="2026-09-11 17:39:14.959411088 +0000 UTC m=+22.847589434"
Sep 11 17:39:14.983362 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:14.983310    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k4gcm" podStartSLOduration=1.605711716 podStartE2EDuration="21.983290757s" totalImagesPullingTime="20.377579041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:14.982827469 +0000 UTC m=+22.871005815" watchObservedRunningTime="2026-09-11 17:39:14.983290757 +0000 UTC m=+22.871469103"
Sep 11 17:39:15.627695 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:15.627665    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:15.627883 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:15.627797    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:15.867137 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:15.867102    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5" event={"ID":"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c","Type":"ContainerStarted","Data":"a8754f2a8e6ccc5cf3e598df4632251ecf42f8048c3abdd623ac7eea87b4e93a"}
Sep 11 17:39:15.868836 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:15.868807    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8rs4d" event={"ID":"32fd6194-d269-4529-a491-ac6e45a0874e","Type":"ContainerStarted","Data":"282982cafe4bbdbf96ef059c611999df65261c1143c3eeb62b7b69f8618eebac"}
Sep 11 17:39:15.870484 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:15.870454    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal" event={"ID":"95d8349af1ea27705737de7491da6301","Type":"ContainerStarted","Data":"727dbcf996fe978a3bb94d1b8f80f4b95149f6f9cc06aa8cc4659a087729448e"}
Sep 11 17:39:15.885147 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:15.885050    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8rs4d" podStartSLOduration=2.599862446 podStartE2EDuration="22.885035028s" totalImagesPullingTime="20.285172582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:15.884192685 +0000 UTC m=+23.772371031" watchObservedRunningTime="2026-09-11 17:39:15.885035028 +0000 UTC m=+23.773213374"
Sep 11 17:39:15.903981 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:15.903932    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-215.ec2.internal" podStartSLOduration=21.90391602 podStartE2EDuration="21.90391602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:15.903667161 +0000 UTC m=+23.791845510" watchObservedRunningTime="2026-09-11 17:39:15.90391602 +0000 UTC m=+23.792094368"
Sep 11 17:39:16.060187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:16.059992    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 17:39:16.572357 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:16.572237    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T17:39:16.060180858Z","UUID":"61f23653-468c-427c-ad21-113be274548f","Handler":null,"Name":"","Endpoint":""}
Sep 11 17:39:16.574229 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:16.573988    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 17:39:16.574229 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:16.574023    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 17:39:16.630344 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:16.630309    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:16.630510 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:16.630437    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:16.630845 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:16.630827    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:16.630945 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:16.630917    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:16.874673 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:16.874580    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs" event={"ID":"0db99c1e-299d-4572-8b5f-fbe7c21e2e73","Type":"ContainerStarted","Data":"beae2a96ad54309e313d43a429749a043dd42cd1e09881087f386efc13a78383"}
Sep 11 17:39:17.628121 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:17.628067    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:17.628341 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:17.628219    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:17.879339 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:17.879304    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs" event={"ID":"0db99c1e-299d-4572-8b5f-fbe7c21e2e73","Type":"ContainerStarted","Data":"3835f93f2b0020ed91d326aff4056175aa2b4323dd2a39d0c1823c96b1af20a9"}
Sep 11 17:39:17.882843 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:17.882820    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5" event={"ID":"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c","Type":"ContainerStarted","Data":"53595c80eca73ee4a2f1464bfc14f53415e91b8448264a76ff775ac11ff71903"}
Sep 11 17:39:17.898075 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:17.898025    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dfgxs" podStartSLOduration=2.061828941 podStartE2EDuration="24.898011258s" totalImagesPullingTime="22.836182317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:17.897721505 +0000 UTC m=+25.785899851" watchObservedRunningTime="2026-09-11 17:39:17.898011258 +0000 UTC m=+25.786189603"
Sep 11 17:39:18.019804 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:18.019765    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:39:18.020426 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:18.020399    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:39:18.627897 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:18.627864    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:18.628073 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:18.627872    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:18.628073 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:18.627997    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:18.628219 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:18.628066    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:18.885899 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:18.885821    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:39:18.886919 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:18.886885    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qccvc"
Sep 11 17:39:19.628430 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:19.628402    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:19.628582 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:19.628507    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:19.890250 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:19.890099    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wm7m" event={"ID":"265ef6f1-4e31-464b-81b6-5ba76a7510bc","Type":"ContainerStarted","Data":"79a357c7af19e421f9b8a1369383b7f6373cef8027d4840b90163246bfda5b7e"}
Sep 11 17:39:19.893815 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:19.893787    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5" event={"ID":"7ad358f4-54dc-4aea-9e7f-3387e4f91b7c","Type":"ContainerStarted","Data":"32e636b39d2cdc29c24e9d322b87a3b15629f022cec0bf36b6db71e6beb7bb6f"}
Sep 11 17:39:19.923000 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:19.922954    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5" podStartSLOduration=6.590566628 podStartE2EDuration="26.922939777s" totalImagesPullingTime="20.332373149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:19.921593026 +0000 UTC m=+27.809771372" watchObservedRunningTime="2026-09-11 17:39:19.922939777 +0000 UTC m=+27.811118122"
Sep 11 17:39:20.627651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:20.627616    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:20.627839 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:20.627724    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:20.627839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:20.627822    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:20.627954 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:20.627931    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:20.898485 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:20.898443    2582 generic.go:356] "Generic (PLEG): container finished" podID="265ef6f1-4e31-464b-81b6-5ba76a7510bc" containerID="79a357c7af19e421f9b8a1369383b7f6373cef8027d4840b90163246bfda5b7e" exitCode=0
Sep 11 17:39:20.900224 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:20.900195    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wm7m" event={"ID":"265ef6f1-4e31-464b-81b6-5ba76a7510bc","Type":"ContainerDied","Data":"79a357c7af19e421f9b8a1369383b7f6373cef8027d4840b90163246bfda5b7e"}
Sep 11 17:39:20.900351 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:20.900251    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:39:20.900748 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:20.900726    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:39:20.900859 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:20.900799    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:39:20.935260 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:20.935233    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:39:20.935568 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:20.935395    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:39:21.621560 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:21.621491    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cvgh7"]
Sep 11 17:39:21.621741 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:21.621682    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:21.624502 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:21.621856    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:21.624502 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:21.624065    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2bq7k"]
Sep 11 17:39:21.624502 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:21.624150    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l458g"]
Sep 11 17:39:21.624502 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:21.624256    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:21.624502 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:21.624466    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:21.624502 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:21.624429    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:21.624917 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:21.624594    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:22.905459 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:22.905425    2582 generic.go:356] "Generic (PLEG): container finished" podID="265ef6f1-4e31-464b-81b6-5ba76a7510bc" containerID="7bc159f6c2cbb6050d5fa5700d5c70ad1b6812e22cb8e25bd19f2491cb2e49c2" exitCode=0
Sep 11 17:39:22.906275 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:22.905511    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wm7m" event={"ID":"265ef6f1-4e31-464b-81b6-5ba76a7510bc","Type":"ContainerDied","Data":"7bc159f6c2cbb6050d5fa5700d5c70ad1b6812e22cb8e25bd19f2491cb2e49c2"}
Sep 11 17:39:23.628609 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:23.628276    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:23.628609 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:23.628278    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:23.628808 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:23.628632    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:23.628808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:23.628316    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:23.628808 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:23.628689    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:23.628808 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:23.628750    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:23.946871 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:23.946847    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-88zgk_1ac45d7c-f7d4-422d-8e43-3cbc853fe1df/dns-node-resolver/0.log"
Sep 11 17:39:24.912421 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:24.912382    2582 generic.go:356] "Generic (PLEG): container finished" podID="265ef6f1-4e31-464b-81b6-5ba76a7510bc" containerID="5df04dd257683b9d203fdecabb8e337096fe5733ce92f771e8ab199d47fadd01" exitCode=0
Sep 11 17:39:24.912563 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:24.912449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wm7m" event={"ID":"265ef6f1-4e31-464b-81b6-5ba76a7510bc","Type":"ContainerDied","Data":"5df04dd257683b9d203fdecabb8e337096fe5733ce92f771e8ab199d47fadd01"}
Sep 11 17:39:25.114030 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:25.114003    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkt44_015e5b25-5f9b-4c49-b6ac-575f4065c9ea/node-ca/0.log"
Sep 11 17:39:25.263259 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:25.263170    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:25.263259 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:25.263221    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:25.263435 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.263311    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:39:25.263435 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.263313    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:39:25.263435 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.263364    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret podName:46c761dc-4a0b-419e-8c89-86f14b88b4a9 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:57.263350882 +0000 UTC m=+65.151529205 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret") pod "global-pull-secret-syncer-cvgh7" (UID: "46c761dc-4a0b-419e-8c89-86f14b88b4a9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:39:25.263435 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.263378    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs podName:1cde1024-eadf-4e40-8354-b153794c1109 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:57.263371361 +0000 UTC m=+65.151549685 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs") pod "network-metrics-daemon-l458g" (UID: "1cde1024-eadf-4e40-8354-b153794c1109") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:39:25.464536 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:25.464501    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4jn\" (UniqueName: \"kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn\") pod \"network-check-target-2bq7k\" (UID: \"1580cdd2-f034-4b8a-a78d-5b41d050d18a\") " pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:25.464710 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.464622    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:39:25.464710 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.464638    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:39:25.464710 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.464649    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4z4jn for pod openshift-network-diagnostics/network-check-target-2bq7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:39:25.464710 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.464698    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn podName:1580cdd2-f034-4b8a-a78d-5b41d050d18a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:57.464685135 +0000 UTC m=+65.352863458 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4z4jn" (UniqueName: "kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn") pod "network-check-target-2bq7k" (UID: "1580cdd2-f034-4b8a-a78d-5b41d050d18a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:39:25.627949 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:25.627867    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:25.627949 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:25.627901    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:25.627949 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:25.627901    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:25.628176 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.627984    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l458g" podUID="1cde1024-eadf-4e40-8354-b153794c1109"
Sep 11 17:39:25.628176 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.628058    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cvgh7" podUID="46c761dc-4a0b-419e-8c89-86f14b88b4a9"
Sep 11 17:39:25.628176 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:39:25.628137    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bq7k" podUID="1580cdd2-f034-4b8a-a78d-5b41d050d18a"
Sep 11 17:39:26.932218 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:26.932189    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-215.ec2.internal" event="NodeReady"
Sep 11 17:39:26.932689 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:26.932335    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 17:39:27.628553 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:27.628472    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:27.628722 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:27.628472    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:27.628722 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:27.628578    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:27.632880 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:27.632490    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 17:39:27.632880 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:27.632515    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 17:39:27.632880 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:27.632496    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 17:39:27.632880 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:27.632496    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hwqhm\""
Sep 11 17:39:27.632880 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:27.632497    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 17:39:27.632880 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:27.632613    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7q2fl\""
Sep 11 17:39:29.930872 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:29.930836    2582 generic.go:356] "Generic (PLEG): container finished" podID="265ef6f1-4e31-464b-81b6-5ba76a7510bc" containerID="b0fe905690d79590d0c52e96571bcabd27e901a5114afc4db426e8aae3ab1a9d" exitCode=0
Sep 11 17:39:29.931437 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:29.930916    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wm7m" event={"ID":"265ef6f1-4e31-464b-81b6-5ba76a7510bc","Type":"ContainerDied","Data":"b0fe905690d79590d0c52e96571bcabd27e901a5114afc4db426e8aae3ab1a9d"}
Sep 11 17:39:30.935752 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:30.935724    2582 generic.go:356] "Generic (PLEG): container finished" podID="265ef6f1-4e31-464b-81b6-5ba76a7510bc" containerID="4a23ac872666441813cd55246f2e122e6fa49c02f4a7f30b40fd9dd0d0c52d64" exitCode=0
Sep 11 17:39:30.936106 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:30.935780    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wm7m" event={"ID":"265ef6f1-4e31-464b-81b6-5ba76a7510bc","Type":"ContainerDied","Data":"4a23ac872666441813cd55246f2e122e6fa49c02f4a7f30b40fd9dd0d0c52d64"}
Sep 11 17:39:31.941421 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:31.941390    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wm7m" event={"ID":"265ef6f1-4e31-464b-81b6-5ba76a7510bc","Type":"ContainerStarted","Data":"fdc1baa21235dac1faaf961700efb62a06de89e2171b5c3a7f7a1907721164b7"}
Sep 11 17:39:31.969534 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:31.969482    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6wm7m" podStartSLOduration=7.305801686 podStartE2EDuration="38.969461679s" totalImagesPullingTime="31.663659993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:31.967413075 +0000 UTC m=+39.855591418" watchObservedRunningTime="2026-09-11 17:39:31.969461679 +0000 UTC m=+39.857640026"
Sep 11 17:39:45.596248 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.596210    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59bbd88f7d-bkxbf"]
Sep 11 17:39:45.617434 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.617395    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59bbd88f7d-bkxbf"]
Sep 11 17:39:45.617590 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.617518    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.618189 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.618163    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2tj6f"]
Sep 11 17:39:45.620128 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.620106    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 17:39:45.620280 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.620262    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 17:39:45.620447 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.620431    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 17:39:45.620550 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.620532    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2cprr\""
Sep 11 17:39:45.627097 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.627062    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 17:39:45.632405 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.632380    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xnbll"]
Sep 11 17:39:45.632714 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.632516    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.635662 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.635639    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zb994\""
Sep 11 17:39:45.635964 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.635944    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 17:39:45.636056 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.635982    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 17:39:45.653714 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.653677    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2tj6f"]
Sep 11 17:39:45.653714 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.653718    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xnbll"]
Sep 11 17:39:45.653913 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.653801    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.656915 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.656892    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 17:39:45.657053 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.656946    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 17:39:45.657053 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.657041    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-72jzj\""
Sep 11 17:39:45.657192 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.657169    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 17:39:45.657328 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.657313    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 17:39:45.716800 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.716766    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bs2vs"]
Sep 11 17:39:45.734062 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.734030    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bs2vs"]
Sep 11 17:39:45.734254 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.734141    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bs2vs"
Sep 11 17:39:45.736938 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.736917    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 17:39:45.737068 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.736942    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-66cbf\""
Sep 11 17:39:45.737068 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.736958    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 17:39:45.737842 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.737826    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 17:39:45.806557 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806515    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flnsx\" (UniqueName: \"kubernetes.io/projected/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-kube-api-access-flnsx\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.806726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-crio-socket\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.806726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806588    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-data-volume\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.806726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806607    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/614f25b1-5b49-4f8c-8e90-af062ab8942e-metrics-tls\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.806726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806632    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20351bca-486b-456e-83b3-8be534dcbf3a-ca-trust-extracted\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.806726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806650    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.806726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806670    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drv6d\" (UniqueName: \"kubernetes.io/projected/614f25b1-5b49-4f8c-8e90-af062ab8942e-kube-api-access-drv6d\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.806726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806697    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/614f25b1-5b49-4f8c-8e90-af062ab8942e-config-volume\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.806952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806731    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/614f25b1-5b49-4f8c-8e90-af062ab8942e-tmp-dir\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.806952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806749    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20351bca-486b-456e-83b3-8be534dcbf3a-image-registry-private-configuration\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.806952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20351bca-486b-456e-83b3-8be534dcbf3a-installation-pull-secrets\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.806952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806804    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.806952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20351bca-486b-456e-83b3-8be534dcbf3a-registry-certificates\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.806952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806863    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20351bca-486b-456e-83b3-8be534dcbf3a-bound-sa-token\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.806952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806891    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20351bca-486b-456e-83b3-8be534dcbf3a-registry-tls\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.806952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806905    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20351bca-486b-456e-83b3-8be534dcbf3a-trusted-ca\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.806952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.806918    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5nmr\" (UniqueName: \"kubernetes.io/projected/20351bca-486b-456e-83b3-8be534dcbf3a-kube-api-access-n5nmr\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.907995 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.907960    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20351bca-486b-456e-83b3-8be534dcbf3a-registry-certificates\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.908212 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20351bca-486b-456e-83b3-8be534dcbf3a-bound-sa-token\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.908212 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20351bca-486b-456e-83b3-8be534dcbf3a-registry-tls\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.908212 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908062    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20351bca-486b-456e-83b3-8be534dcbf3a-trusted-ca\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.908212 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908107    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5nmr\" (UniqueName: \"kubernetes.io/projected/20351bca-486b-456e-83b3-8be534dcbf3a-kube-api-access-n5nmr\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.908212 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flnsx\" (UniqueName: \"kubernetes.io/projected/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-kube-api-access-flnsx\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.908212 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908173    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-crio-socket\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.908212 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908193    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-data-volume\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.908562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908321    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/614f25b1-5b49-4f8c-8e90-af062ab8942e-metrics-tls\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.908562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908356    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20351bca-486b-456e-83b3-8be534dcbf3a-ca-trust-extracted\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.908562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.908562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908395    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-crio-socket\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.908562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908407    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drv6d\" (UniqueName: \"kubernetes.io/projected/614f25b1-5b49-4f8c-8e90-af062ab8942e-kube-api-access-drv6d\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.908562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908480    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8l2p\" (UniqueName: \"kubernetes.io/projected/5614e759-35f7-4848-9eb6-f1fa63aba1ba-kube-api-access-n8l2p\") pod \"ingress-canary-bs2vs\" (UID: \"5614e759-35f7-4848-9eb6-f1fa63aba1ba\") " pod="openshift-ingress-canary/ingress-canary-bs2vs"
Sep 11 17:39:45.908562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5614e759-35f7-4848-9eb6-f1fa63aba1ba-cert\") pod \"ingress-canary-bs2vs\" (UID: \"5614e759-35f7-4848-9eb6-f1fa63aba1ba\") " pod="openshift-ingress-canary/ingress-canary-bs2vs"
Sep 11 17:39:45.908562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908508    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-data-volume\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.908562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908536    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/614f25b1-5b49-4f8c-8e90-af062ab8942e-config-volume\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.908562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908559    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/614f25b1-5b49-4f8c-8e90-af062ab8942e-tmp-dir\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.909030 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908585    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20351bca-486b-456e-83b3-8be534dcbf3a-image-registry-private-configuration\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.909030 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908681    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20351bca-486b-456e-83b3-8be534dcbf3a-ca-trust-extracted\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.909030 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908744    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20351bca-486b-456e-83b3-8be534dcbf3a-installation-pull-secrets\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.909030 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908792    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.909030 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.908944    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/614f25b1-5b49-4f8c-8e90-af062ab8942e-tmp-dir\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.909305 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.909150    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/614f25b1-5b49-4f8c-8e90-af062ab8942e-config-volume\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.909689 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.909501    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.909689 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.909626    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20351bca-486b-456e-83b3-8be534dcbf3a-registry-certificates\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.909689 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.909628    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20351bca-486b-456e-83b3-8be534dcbf3a-trusted-ca\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.912731 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.912710    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20351bca-486b-456e-83b3-8be534dcbf3a-image-registry-private-configuration\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.912816 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.912754    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20351bca-486b-456e-83b3-8be534dcbf3a-installation-pull-secrets\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.912816 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.912796    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.912916 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.912897    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20351bca-486b-456e-83b3-8be534dcbf3a-registry-tls\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.913042 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.913029    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/614f25b1-5b49-4f8c-8e90-af062ab8942e-metrics-tls\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.917048 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.917021    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20351bca-486b-456e-83b3-8be534dcbf3a-bound-sa-token\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.917703 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.917683    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5nmr\" (UniqueName: \"kubernetes.io/projected/20351bca-486b-456e-83b3-8be534dcbf3a-kube-api-access-n5nmr\") pod \"image-registry-59bbd88f7d-bkxbf\" (UID: \"20351bca-486b-456e-83b3-8be534dcbf3a\") " pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.918473 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.918454    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flnsx\" (UniqueName: \"kubernetes.io/projected/b16d21f2-b86b-4cb0-9c06-00bc7cb53af0-kube-api-access-flnsx\") pod \"insights-runtime-extractor-xnbll\" (UID: \"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0\") " pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:45.920528 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.920511    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drv6d\" (UniqueName: \"kubernetes.io/projected/614f25b1-5b49-4f8c-8e90-af062ab8942e-kube-api-access-drv6d\") pod \"dns-default-2tj6f\" (UID: \"614f25b1-5b49-4f8c-8e90-af062ab8942e\") " pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.927743 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.927719    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:45.941627 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.941595    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:45.962526 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:45.962499    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xnbll"
Sep 11 17:39:46.009602 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.009570    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8l2p\" (UniqueName: \"kubernetes.io/projected/5614e759-35f7-4848-9eb6-f1fa63aba1ba-kube-api-access-n8l2p\") pod \"ingress-canary-bs2vs\" (UID: \"5614e759-35f7-4848-9eb6-f1fa63aba1ba\") " pod="openshift-ingress-canary/ingress-canary-bs2vs"
Sep 11 17:39:46.009750 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.009608    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5614e759-35f7-4848-9eb6-f1fa63aba1ba-cert\") pod \"ingress-canary-bs2vs\" (UID: \"5614e759-35f7-4848-9eb6-f1fa63aba1ba\") " pod="openshift-ingress-canary/ingress-canary-bs2vs"
Sep 11 17:39:46.012120 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.012063    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5614e759-35f7-4848-9eb6-f1fa63aba1ba-cert\") pod \"ingress-canary-bs2vs\" (UID: \"5614e759-35f7-4848-9eb6-f1fa63aba1ba\") " pod="openshift-ingress-canary/ingress-canary-bs2vs"
Sep 11 17:39:46.018163 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.018142    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8l2p\" (UniqueName: \"kubernetes.io/projected/5614e759-35f7-4848-9eb6-f1fa63aba1ba-kube-api-access-n8l2p\") pod \"ingress-canary-bs2vs\" (UID: \"5614e759-35f7-4848-9eb6-f1fa63aba1ba\") " pod="openshift-ingress-canary/ingress-canary-bs2vs"
Sep 11 17:39:46.042915 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.042620    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bs2vs"
Sep 11 17:39:46.262035 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.262004    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2tj6f"]
Sep 11 17:39:46.262866 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.262844    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59bbd88f7d-bkxbf"]
Sep 11 17:39:46.263667 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.263647    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xnbll"]
Sep 11 17:39:46.264633 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:39:46.264610    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614f25b1_5b49_4f8c_8e90_af062ab8942e.slice/crio-8e0a2c7c724352a4523bd8cbaaa7b6da08be95cbc18338199c0e1baf8e71eea3 WatchSource:0}: Error finding container 8e0a2c7c724352a4523bd8cbaaa7b6da08be95cbc18338199c0e1baf8e71eea3: Status 404 returned error can't find the container with id 8e0a2c7c724352a4523bd8cbaaa7b6da08be95cbc18338199c0e1baf8e71eea3
Sep 11 17:39:46.264831 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:39:46.264818    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20351bca_486b_456e_83b3_8be534dcbf3a.slice/crio-f0b855cbc95571f4782ac13b93b6e63e129a49fd0ca158a6547a2c106869bc9d WatchSource:0}: Error finding container f0b855cbc95571f4782ac13b93b6e63e129a49fd0ca158a6547a2c106869bc9d: Status 404 returned error can't find the container with id f0b855cbc95571f4782ac13b93b6e63e129a49fd0ca158a6547a2c106869bc9d
Sep 11 17:39:46.265909 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:39:46.265891    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb16d21f2_b86b_4cb0_9c06_00bc7cb53af0.slice/crio-d44230ab19083de9d7c8743376f817058104f6673b4e9d0c2c6eba7dd16ec026 WatchSource:0}: Error finding container d44230ab19083de9d7c8743376f817058104f6673b4e9d0c2c6eba7dd16ec026: Status 404 returned error can't find the container with id d44230ab19083de9d7c8743376f817058104f6673b4e9d0c2c6eba7dd16ec026
Sep 11 17:39:46.326475 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.326451    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bs2vs"]
Sep 11 17:39:46.328647 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:39:46.328623    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5614e759_35f7_4848_9eb6_f1fa63aba1ba.slice/crio-602884800ab26f295b99310ccacdaa1d17c7c0ff8eac2ae6512f7677753947d5 WatchSource:0}: Error finding container 602884800ab26f295b99310ccacdaa1d17c7c0ff8eac2ae6512f7677753947d5: Status 404 returned error can't find the container with id 602884800ab26f295b99310ccacdaa1d17c7c0ff8eac2ae6512f7677753947d5
Sep 11 17:39:46.978757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.978642    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xnbll" event={"ID":"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0","Type":"ContainerStarted","Data":"9a602244dfc0d6ed1fff0c7fcd98c3c48ce114a71d93b5d574d4288c86de36d9"}
Sep 11 17:39:46.978757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.978713    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xnbll" event={"ID":"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0","Type":"ContainerStarted","Data":"d44230ab19083de9d7c8743376f817058104f6673b4e9d0c2c6eba7dd16ec026"}
Sep 11 17:39:46.980875 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.980843    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2tj6f" event={"ID":"614f25b1-5b49-4f8c-8e90-af062ab8942e","Type":"ContainerStarted","Data":"8e0a2c7c724352a4523bd8cbaaa7b6da08be95cbc18338199c0e1baf8e71eea3"}
Sep 11 17:39:46.982316 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.982291    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bs2vs" event={"ID":"5614e759-35f7-4848-9eb6-f1fa63aba1ba","Type":"ContainerStarted","Data":"602884800ab26f295b99310ccacdaa1d17c7c0ff8eac2ae6512f7677753947d5"}
Sep 11 17:39:46.984513 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.984026    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf" event={"ID":"20351bca-486b-456e-83b3-8be534dcbf3a","Type":"ContainerStarted","Data":"d1447f850a3a0df907a9d1417ef35bc4df50525c8df3a4cd36dc40f5e1a82731"}
Sep 11 17:39:46.984513 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.984067    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf" event={"ID":"20351bca-486b-456e-83b3-8be534dcbf3a","Type":"ContainerStarted","Data":"f0b855cbc95571f4782ac13b93b6e63e129a49fd0ca158a6547a2c106869bc9d"}
Sep 11 17:39:46.984513 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:46.984284    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:39:47.011433 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:47.011391    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf" podStartSLOduration=2.011374652 podStartE2EDuration="2.011374652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:47.01014373 +0000 UTC m=+54.898322079" watchObservedRunningTime="2026-09-11 17:39:47.011374652 +0000 UTC m=+54.899553010"
Sep 11 17:39:47.989878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:47.989840    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xnbll" event={"ID":"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0","Type":"ContainerStarted","Data":"4fed2333ab3d39635577d5880a594686af59d6d13ab6090388c44d3c895beeb4"}
Sep 11 17:39:49.997245 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:49.997205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xnbll" event={"ID":"b16d21f2-b86b-4cb0-9c06-00bc7cb53af0","Type":"ContainerStarted","Data":"733e4f8f9cbc0d8f907603a7e5aa337ae960ced6177bf588eb37066c4cec6a1e"}
Sep 11 17:39:49.999122 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:49.999092    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2tj6f" event={"ID":"614f25b1-5b49-4f8c-8e90-af062ab8942e","Type":"ContainerStarted","Data":"475052d7fba9ca3fe1f0d24309119e2a9f7c8e8f3b764e6a32ba8606ee9bc7c1"}
Sep 11 17:39:49.999247 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:49.999153    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:39:49.999247 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:49.999181    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2tj6f" event={"ID":"614f25b1-5b49-4f8c-8e90-af062ab8942e","Type":"ContainerStarted","Data":"6d44e7e624a81ea11e9d326b020f6aa7dcc17fae89ffb473d9aad329aaf97b97"}
Sep 11 17:39:50.000529 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:50.000510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bs2vs" event={"ID":"5614e759-35f7-4848-9eb6-f1fa63aba1ba","Type":"ContainerStarted","Data":"8198984387d376b2bb4119f8fadbaf27cd6b92040cb3aead84f23cc9bd75656d"}
Sep 11 17:39:50.019193 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:50.019142    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xnbll" podStartSLOduration=2.305553352 podStartE2EDuration="5.019128031s" totalImagesPullingTime="2.713574679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:39:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:50.017622838 +0000 UTC m=+57.905801186" watchObservedRunningTime="2026-09-11 17:39:50.019128031 +0000 UTC m=+57.907306397"
Sep 11 17:39:50.034764 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:50.034712    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bs2vs" podStartSLOduration=2.219494906 podStartE2EDuration="5.034697273s" totalImagesPullingTime="2.815202367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:50.034043142 +0000 UTC m=+57.922221489" watchObservedRunningTime="2026-09-11 17:39:50.034697273 +0000 UTC m=+57.922875619"
Sep 11 17:39:50.058455 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:50.058404    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2tj6f" podStartSLOduration=2.184075071 podStartE2EDuration="5.058386113s" totalImagesPullingTime="2.874311042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:50.0576844 +0000 UTC m=+57.945862746" watchObservedRunningTime="2026-09-11 17:39:50.058386113 +0000 UTC m=+57.946564450"
Sep 11 17:39:52.919178 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:52.919146    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k8kz5"
Sep 11 17:39:53.746170 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.746132    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f57fb7d77-z4q5r"]
Sep 11 17:39:53.749032 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.749016    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.753564 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.753534    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-569cd\""
Sep 11 17:39:53.753700 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.753566    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 17:39:53.753700 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.753601    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 17:39:53.753700 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.753601    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 17:39:53.753700 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.753602    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 17:39:53.754194 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.754173    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 17:39:53.754194 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.754184    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 17:39:53.754328 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.754238    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 17:39:53.764771 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.764749    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f57fb7d77-z4q5r"]
Sep 11 17:39:53.770227 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.770208    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-oauth-serving-cert\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.770314 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.770292    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-serving-cert\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.770351 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.770314    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-config\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.770351 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.770330    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2x6s\" (UniqueName: \"kubernetes.io/projected/e5d41f55-083c-493b-ac64-8c23aae3bfd0-kube-api-access-m2x6s\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.770424 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.770356    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-oauth-config\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.770424 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.770377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-service-ca\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.870673 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.870633    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-serving-cert\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.870856 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.870672    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-config\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.870856 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.870715    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2x6s\" (UniqueName: \"kubernetes.io/projected/e5d41f55-083c-493b-ac64-8c23aae3bfd0-kube-api-access-m2x6s\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.870856 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.870744    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-oauth-config\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.870856 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.870763    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-service-ca\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.870856 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.870807    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-oauth-serving-cert\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.871601 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.871528    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-oauth-serving-cert\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.871601 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.871527    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-service-ca\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.871806 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.871744    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-config\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.873325 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.873299    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-serving-cert\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.873325 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.873317    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-oauth-config\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:53.879482 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:53.879463    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2x6s\" (UniqueName: \"kubernetes.io/projected/e5d41f55-083c-493b-ac64-8c23aae3bfd0-kube-api-access-m2x6s\") pod \"console-5f57fb7d77-z4q5r\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") " pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:54.057626 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:54.057530    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:39:54.329481 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:54.329408    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f57fb7d77-z4q5r"]
Sep 11 17:39:54.332899 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:39:54.332873    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d41f55_083c_493b_ac64_8c23aae3bfd0.slice/crio-ac101495d1c04737d4934db9a428cfe217d1a498b84694a39fc1dd47d6db4120 WatchSource:0}: Error finding container ac101495d1c04737d4934db9a428cfe217d1a498b84694a39fc1dd47d6db4120: Status 404 returned error can't find the container with id ac101495d1c04737d4934db9a428cfe217d1a498b84694a39fc1dd47d6db4120
Sep 11 17:39:55.019894 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:55.019849    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f57fb7d77-z4q5r" event={"ID":"e5d41f55-083c-493b-ac64-8c23aae3bfd0","Type":"ContainerStarted","Data":"ac101495d1c04737d4934db9a428cfe217d1a498b84694a39fc1dd47d6db4120"}
Sep 11 17:39:57.298376 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.298323    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:57.298752 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.298431    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:57.301540 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.301523    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 17:39:57.301596 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.301579    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 17:39:57.311397 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.311376    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cde1024-eadf-4e40-8354-b153794c1109-metrics-certs\") pod \"network-metrics-daemon-l458g\" (UID: \"1cde1024-eadf-4e40-8354-b153794c1109\") " pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:57.311465 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.311411    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46c761dc-4a0b-419e-8c89-86f14b88b4a9-original-pull-secret\") pod \"global-pull-secret-syncer-cvgh7\" (UID: \"46c761dc-4a0b-419e-8c89-86f14b88b4a9\") " pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:57.340775 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.340753    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cvgh7"
Sep 11 17:39:57.355371 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.355349    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7q2fl\""
Sep 11 17:39:57.362800 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.362777    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l458g"
Sep 11 17:39:57.500060 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.500032    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4jn\" (UniqueName: \"kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn\") pod \"network-check-target-2bq7k\" (UID: \"1580cdd2-f034-4b8a-a78d-5b41d050d18a\") " pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:57.502908 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.502891    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 17:39:57.513799 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.513782    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 17:39:57.524425 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.524403    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z4jn\" (UniqueName: \"kubernetes.io/projected/1580cdd2-f034-4b8a-a78d-5b41d050d18a-kube-api-access-4z4jn\") pod \"network-check-target-2bq7k\" (UID: \"1580cdd2-f034-4b8a-a78d-5b41d050d18a\") " pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:57.609800 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.609728    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cvgh7"]
Sep 11 17:39:57.612530 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:39:57.612502    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c761dc_4a0b_419e_8c89_86f14b88b4a9.slice/crio-cb8841a0d83717a529d9373429e1fdd140dd5292d261d52526fdd4f220079feb WatchSource:0}: Error finding container cb8841a0d83717a529d9373429e1fdd140dd5292d261d52526fdd4f220079feb: Status 404 returned error can't find the container with id cb8841a0d83717a529d9373429e1fdd140dd5292d261d52526fdd4f220079feb
Sep 11 17:39:57.641160 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.641127    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l458g"]
Sep 11 17:39:57.643743 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:39:57.643723    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cde1024_eadf_4e40_8354_b153794c1109.slice/crio-263f2f9351362242914d59046576b9368f7f8062bb1441e9bf2527639b7ad815 WatchSource:0}: Error finding container 263f2f9351362242914d59046576b9368f7f8062bb1441e9bf2527639b7ad815: Status 404 returned error can't find the container with id 263f2f9351362242914d59046576b9368f7f8062bb1441e9bf2527639b7ad815
Sep 11 17:39:57.649452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.649433    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hwqhm\""
Sep 11 17:39:57.657704 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.657686    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:39:57.924179 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:57.924147    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2bq7k"]
Sep 11 17:39:57.927701 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:39:57.927676    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1580cdd2_f034_4b8a_a78d_5b41d050d18a.slice/crio-34451f7068505b7f46512715f650ffc3ddb5d8db2c78b4d04ec34dc2194d5302 WatchSource:0}: Error finding container 34451f7068505b7f46512715f650ffc3ddb5d8db2c78b4d04ec34dc2194d5302: Status 404 returned error can't find the container with id 34451f7068505b7f46512715f650ffc3ddb5d8db2c78b4d04ec34dc2194d5302
Sep 11 17:39:58.031400 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.031367    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2bq7k" event={"ID":"1580cdd2-f034-4b8a-a78d-5b41d050d18a","Type":"ContainerStarted","Data":"34451f7068505b7f46512715f650ffc3ddb5d8db2c78b4d04ec34dc2194d5302"}
Sep 11 17:39:58.032475 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.032455    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cvgh7" event={"ID":"46c761dc-4a0b-419e-8c89-86f14b88b4a9","Type":"ContainerStarted","Data":"cb8841a0d83717a529d9373429e1fdd140dd5292d261d52526fdd4f220079feb"}
Sep 11 17:39:58.034022 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.034003    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f57fb7d77-z4q5r" event={"ID":"e5d41f55-083c-493b-ac64-8c23aae3bfd0","Type":"ContainerStarted","Data":"57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54"}
Sep 11 17:39:58.035178 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.035160    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l458g" event={"ID":"1cde1024-eadf-4e40-8354-b153794c1109","Type":"ContainerStarted","Data":"263f2f9351362242914d59046576b9368f7f8062bb1441e9bf2527639b7ad815"}
Sep 11 17:39:58.054494 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.054445    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f57fb7d77-z4q5r" podStartSLOduration=2.083008014 podStartE2EDuration="5.054428014s" totalImagesPullingTime="2.97142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:58.052830655 +0000 UTC m=+65.941009001" watchObservedRunningTime="2026-09-11 17:39:58.054428014 +0000 UTC m=+65.942606360"
Sep 11 17:39:58.793194 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.792644    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"]
Sep 11 17:39:58.815492 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.815460    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"]
Sep 11 17:39:58.815642 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.815620    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:58.819725 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.819694    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 17:39:58.821009 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.819980    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 17:39:58.821009 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.820235    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 17:39:58.821009 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.820236    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9cgfn\""
Sep 11 17:39:58.821009 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.820463    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 17:39:58.821009 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.820540    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 17:39:58.832921 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.832890    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pqc9d"]
Sep 11 17:39:58.851828 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.851797    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:58.854764 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.854737    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 17:39:58.854910 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.854812    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m696x\""
Sep 11 17:39:58.855039 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.855016    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 17:39:58.855215 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.855191    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 17:39:58.912409 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.912265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28nrb\" (UniqueName: \"kubernetes.io/projected/bd35d437-91ee-45e7-b74e-3860a275e4a3-kube-api-access-28nrb\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:58.912409 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.912319    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90ebc72c-0d00-404f-b352-b17a0d410739-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:58.912715 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.912687    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd35d437-91ee-45e7-b74e-3860a275e4a3-sys\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:58.912837 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.912762    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90ebc72c-0d00-404f-b352-b17a0d410739-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:58.912837 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.912808    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90ebc72c-0d00-404f-b352-b17a0d410739-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:58.912942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.912834    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:58.912942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.912862    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd35d437-91ee-45e7-b74e-3860a275e4a3-root\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:58.912942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.912921    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd35d437-91ee-45e7-b74e-3860a275e4a3-metrics-client-ca\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:58.913112 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.913000    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-tls\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:58.913112 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.913034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-wtmp\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:58.913112 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.913067    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng876\" (UniqueName: \"kubernetes.io/projected/90ebc72c-0d00-404f-b352-b17a0d410739-kube-api-access-ng876\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:58.913253 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.913109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-textfile\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:58.913253 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:58.913140    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.014414 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014368    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd35d437-91ee-45e7-b74e-3860a275e4a3-root\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.014414 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd35d437-91ee-45e7-b74e-3860a275e4a3-metrics-client-ca\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.014675 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014446    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-tls\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.014675 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-wtmp\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.014675 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng876\" (UniqueName: \"kubernetes.io/projected/90ebc72c-0d00-404f-b352-b17a0d410739-kube-api-access-ng876\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:59.014675 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014501    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd35d437-91ee-45e7-b74e-3860a275e4a3-root\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.014675 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014526    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-textfile\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.014675 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014585    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.014675 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014636    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28nrb\" (UniqueName: \"kubernetes.io/projected/bd35d437-91ee-45e7-b74e-3860a275e4a3-kube-api-access-28nrb\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.014675 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014665    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-wtmp\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.015065 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90ebc72c-0d00-404f-b352-b17a0d410739-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:59.015065 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014727    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd35d437-91ee-45e7-b74e-3860a275e4a3-sys\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.015065 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014777    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90ebc72c-0d00-404f-b352-b17a0d410739-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:59.015065 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90ebc72c-0d00-404f-b352-b17a0d410739-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:59.015065 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014847    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.015065 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014890    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-textfile\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.015065 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.014956    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd35d437-91ee-45e7-b74e-3860a275e4a3-sys\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.015447 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.015150    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd35d437-91ee-45e7-b74e-3860a275e4a3-metrics-client-ca\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.015447 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.015366    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.015928 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.015855    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90ebc72c-0d00-404f-b352-b17a0d410739-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:59.017874 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.017848    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-tls\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.017976 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.017925    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd35d437-91ee-45e7-b74e-3860a275e4a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.018045 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.018018    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90ebc72c-0d00-404f-b352-b17a0d410739-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:59.018160 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.018138    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90ebc72c-0d00-404f-b352-b17a0d410739-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:59.024372 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.024344    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng876\" (UniqueName: \"kubernetes.io/projected/90ebc72c-0d00-404f-b352-b17a0d410739-kube-api-access-ng876\") pod \"openshift-state-metrics-67dcf8c979-8z4dg\" (UID: \"90ebc72c-0d00-404f-b352-b17a0d410739\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:59.024717 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.024666    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28nrb\" (UniqueName: \"kubernetes.io/projected/bd35d437-91ee-45e7-b74e-3860a275e4a3-kube-api-access-28nrb\") pod \"node-exporter-pqc9d\" (UID: \"bd35d437-91ee-45e7-b74e-3860a275e4a3\") " pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.130194 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.130116    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"
Sep 11 17:39:59.163242 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.163208    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pqc9d"
Sep 11 17:39:59.305225 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:39:59.305112    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd35d437_91ee_45e7_b74e_3860a275e4a3.slice/crio-849e0d5b2cc26c89c32c412dbda9801fe9e4337d7686c8acfc15fc1c6d27efde WatchSource:0}: Error finding container 849e0d5b2cc26c89c32c412dbda9801fe9e4337d7686c8acfc15fc1c6d27efde: Status 404 returned error can't find the container with id 849e0d5b2cc26c89c32c412dbda9801fe9e4337d7686c8acfc15fc1c6d27efde
Sep 11 17:39:59.593020 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:39:59.592990    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg"]
Sep 11 17:39:59.597589 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:39:59.597542    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ebc72c_0d00_404f_b352_b17a0d410739.slice/crio-128fb510ebe7a4fe97b16df59770433229a28daad2250d0f807b6c6907b2f8e1 WatchSource:0}: Error finding container 128fb510ebe7a4fe97b16df59770433229a28daad2250d0f807b6c6907b2f8e1: Status 404 returned error can't find the container with id 128fb510ebe7a4fe97b16df59770433229a28daad2250d0f807b6c6907b2f8e1
Sep 11 17:40:00.006969 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:00.006889    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2tj6f"
Sep 11 17:40:00.047873 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:00.047742    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg" event={"ID":"90ebc72c-0d00-404f-b352-b17a0d410739","Type":"ContainerStarted","Data":"7d4f0938e83553948464af1f2e956575a97835676127a7285ebdc8cc8088ae6c"}
Sep 11 17:40:00.047873 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:00.047809    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg" event={"ID":"90ebc72c-0d00-404f-b352-b17a0d410739","Type":"ContainerStarted","Data":"6bb6ee506a7b807dbfd7c122a71a420afd6357c610b13a8014d61963a07741dc"}
Sep 11 17:40:00.047873 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:00.047831    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg" event={"ID":"90ebc72c-0d00-404f-b352-b17a0d410739","Type":"ContainerStarted","Data":"128fb510ebe7a4fe97b16df59770433229a28daad2250d0f807b6c6907b2f8e1"}
Sep 11 17:40:00.050713 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:00.050628    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pqc9d" event={"ID":"bd35d437-91ee-45e7-b74e-3860a275e4a3","Type":"ContainerStarted","Data":"849e0d5b2cc26c89c32c412dbda9801fe9e4337d7686c8acfc15fc1c6d27efde"}
Sep 11 17:40:00.055996 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:00.055888    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l458g" event={"ID":"1cde1024-eadf-4e40-8354-b153794c1109","Type":"ContainerStarted","Data":"6b1cccae129245d237b717a6322f8d7f8802410502dbf8c9ea2347255fd9ff83"}
Sep 11 17:40:00.055996 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:00.055958    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l458g" event={"ID":"1cde1024-eadf-4e40-8354-b153794c1109","Type":"ContainerStarted","Data":"4f54eabf38baaf708cf11387ec2873a8a0b6e168c8016ec29419ed31c77ac7d8"}
Sep 11 17:40:01.802944 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.802880    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l458g" podStartSLOduration=67.144745384 podStartE2EDuration="1m8.802855451s" totalImagesPullingTime="1.658110067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:40:00.073637531 +0000 UTC m=+67.961815878" watchObservedRunningTime="2026-09-11 17:40:01.802855451 +0000 UTC m=+69.691033798"
Sep 11 17:40:01.804105 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.804065    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-d7fc48577-vzxsx"]
Sep 11 17:40:01.807093 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.807057    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.811496 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.811325    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2ou2m56d01jjk\""
Sep 11 17:40:01.811496 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.811405    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 17:40:01.811670 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.811628    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 17:40:01.811670 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.811631    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nbvsw\""
Sep 11 17:40:01.812550 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.812528    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 17:40:01.812666 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.812527    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 17:40:01.813105 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.813069    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 17:40:01.827525 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.825948    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d7fc48577-vzxsx"]
Sep 11 17:40:01.839664 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.839621    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-grpc-tls\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.839839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.839679    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.839839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.839719    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl2qr\" (UniqueName: \"kubernetes.io/projected/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-kube-api-access-wl2qr\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.839839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.839785    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.839839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.839809    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-tls\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.839839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.839838    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-metrics-client-ca\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.840065 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.839864    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.840065 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.839934    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.940690 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.940658    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.940857 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.940719    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-grpc-tls\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.940857 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.940745    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.940857 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.940767    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl2qr\" (UniqueName: \"kubernetes.io/projected/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-kube-api-access-wl2qr\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.940857 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.940794    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.940857 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.940819    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-tls\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.940857 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.940851    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-metrics-client-ca\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.941172 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.940880    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.941840 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.941814    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-metrics-client-ca\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.943877 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.943854    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.944280 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.944256    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.944443 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.944416    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.944521 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.944474    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-tls\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.944810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.944789    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.944871 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.944856    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-secret-grpc-tls\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:01.952007 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:01.951984    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl2qr\" (UniqueName: \"kubernetes.io/projected/9a68b63b-ee60-4ddb-85b9-7129ddd885dc-kube-api-access-wl2qr\") pod \"thanos-querier-d7fc48577-vzxsx\" (UID: \"9a68b63b-ee60-4ddb-85b9-7129ddd885dc\") " pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:02.118620 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:02.118517    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:03.392863 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:03.392838    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d7fc48577-vzxsx"]
Sep 11 17:40:03.394734 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:40:03.394707    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a68b63b_ee60_4ddb_85b9_7129ddd885dc.slice/crio-6ae2a64928609ef82fc7732d262a1bfe281424affefa38c58287f49d11c6cb8f WatchSource:0}: Error finding container 6ae2a64928609ef82fc7732d262a1bfe281424affefa38c58287f49d11c6cb8f: Status 404 returned error can't find the container with id 6ae2a64928609ef82fc7732d262a1bfe281424affefa38c58287f49d11c6cb8f
Sep 11 17:40:04.058323 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.058286    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:40:04.058511 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.058350    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:40:04.063192 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.063165    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:40:04.070683 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.070654    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx" event={"ID":"9a68b63b-ee60-4ddb-85b9-7129ddd885dc","Type":"ContainerStarted","Data":"6ae2a64928609ef82fc7732d262a1bfe281424affefa38c58287f49d11c6cb8f"}
Sep 11 17:40:04.072627 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.072602    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg" event={"ID":"90ebc72c-0d00-404f-b352-b17a0d410739","Type":"ContainerStarted","Data":"642ef0a380c4284e9cff8d4ccec06bd4a4f6f682c59c895697ddb8a0b839861b"}
Sep 11 17:40:04.074180 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.074160    2582 generic.go:356] "Generic (PLEG): container finished" podID="bd35d437-91ee-45e7-b74e-3860a275e4a3" containerID="14221f1c8caef7d243d5bea46ac793c95037206a73e52c9d28f5233b9657af8b" exitCode=0
Sep 11 17:40:04.074262 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.074188    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pqc9d" event={"ID":"bd35d437-91ee-45e7-b74e-3860a275e4a3","Type":"ContainerDied","Data":"14221f1c8caef7d243d5bea46ac793c95037206a73e52c9d28f5233b9657af8b"}
Sep 11 17:40:04.075709 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.075688    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2bq7k" event={"ID":"1580cdd2-f034-4b8a-a78d-5b41d050d18a","Type":"ContainerStarted","Data":"8a9ccab0e58cb450e1d0f5269104c93b7b127be226749e5cab2e34d994c8a4b6"}
Sep 11 17:40:04.075806 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.075789    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:40:04.077234 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.077212    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cvgh7" event={"ID":"46c761dc-4a0b-419e-8c89-86f14b88b4a9","Type":"ContainerStarted","Data":"3d781cb42e62dbc24f7b1c353a5d4e7524db3167f4333d6afaefaad179991967"}
Sep 11 17:40:04.081195 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.081174    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:40:04.128407 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.128355    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2bq7k" podStartSLOduration=65.960324239 podStartE2EDuration="1m11.128336727s" totalImagesPullingTime="5.168012488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:40:04.127226594 +0000 UTC m=+72.015404952" watchObservedRunningTime="2026-09-11 17:40:04.128336727 +0000 UTC m=+72.016515072"
Sep 11 17:40:04.144560 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.144498    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cvgh7" podStartSLOduration=65.661012541 podStartE2EDuration="1m11.144473824s" totalImagesPullingTime="5.483461283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:40:04.143178863 +0000 UTC m=+72.031357210" watchObservedRunningTime="2026-09-11 17:40:04.144473824 +0000 UTC m=+72.032652184"
Sep 11 17:40:04.167911 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:04.167859    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8z4dg" podStartSLOduration=2.914827437 podStartE2EDuration="6.167841995s" totalImagesPullingTime="3.253014558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:40:04.166022058 +0000 UTC m=+72.054200406" watchObservedRunningTime="2026-09-11 17:40:04.167841995 +0000 UTC m=+72.056020340"
Sep 11 17:40:05.083762 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.083720    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pqc9d" event={"ID":"bd35d437-91ee-45e7-b74e-3860a275e4a3","Type":"ContainerStarted","Data":"495ca4a251c3a95355c60ad49f0ab56ef321185a74d3fcc25f023bdcf9bf7a25"}
Sep 11 17:40:05.084207 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.083782    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pqc9d" event={"ID":"bd35d437-91ee-45e7-b74e-3860a275e4a3","Type":"ContainerStarted","Data":"305a644500ecd1723b06c0096d60e1cfa2579cd7e81baa5c8813583700f86215"}
Sep 11 17:40:05.175278 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.175224    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pqc9d" podStartSLOduration=3.308869065 podStartE2EDuration="7.17520804s" totalImagesPullingTime="3.789378906s" totalInitContainerRuntime="76.960069ms" isStatefulPod=true podCreationTimestamp="2026-09-11 17:39:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:40:05.120896826 +0000 UTC m=+73.009075173" watchObservedRunningTime="2026-09-11 17:40:05.17520804 +0000 UTC m=+73.063386387"
Sep 11 17:40:05.175998 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.175976    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 17:40:05.178978 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.178957    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.182136 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.182114    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 17:40:05.182622 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.182607    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a2eb7dbhgc9vf\""
Sep 11 17:40:05.182901 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.182885    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m2mg5\""
Sep 11 17:40:05.183364 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.183351    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 17:40:05.185478 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.185336    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 17:40:05.185478 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.185335    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 17:40:05.185478 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.185396    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 17:40:05.185478 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.185405    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 17:40:05.185760 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.185608    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 17:40:05.185760 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.185665    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 17:40:05.185760 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.185737    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 17:40:05.186723 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.186702    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 17:40:05.186964 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.186946    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 17:40:05.187039 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.186946    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 17:40:05.187039 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.186995    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 17:40:05.187039 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.187007    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 17:40:05.188161 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.188142    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 17:40:05.216689 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.216651    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 17:40:05.270091 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270054    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config-out\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270258 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270111    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270258 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270136    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270258 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270258 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270179    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270258 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w445\" (UniqueName: \"kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-kube-api-access-6w445\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270258 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270216    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-web-config\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270258 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270233    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270468 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270269    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270468 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270291    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270468 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270312    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270468 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270331    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270468 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270345    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270468 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270385    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270698 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270476    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270698 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270698 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270517    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270698 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270538    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270698 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270581    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.270698 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.270606    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371454 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371362    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371454 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371400    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371454 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371422    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371464    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371489    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371512    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371540    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371564    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371631    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config-out\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371652    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371676    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.371726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372261 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371740    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372261 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371762    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w445\" (UniqueName: \"kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-kube-api-access-6w445\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372261 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371790    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-web-config\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372261 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372261 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371844    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372261 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371872    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372261 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.371893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372610 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.372405    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372656 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.372606    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372772 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.372742    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372953 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.372924    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.372953 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.372942    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.373753 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.373731    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.374527 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.374322    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.374898 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.374832    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.375430 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.375399    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.375518 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.375460    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.375897 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.375870    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.376699 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.376650    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.377344 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.377161    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.377344 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.377227    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.377344 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.377310    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.377560 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.377529    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.377878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.377858    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.378276 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.378258    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config-out\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.378679 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.378657    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-web-config\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.387332 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.387299    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w445\" (UniqueName: \"kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-kube-api-access-6w445\") pod \"prometheus-k8s-0\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.487969 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.487906    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:05.778740 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.778706    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 17:40:05.780736 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:40:05.780711    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac9b48d5_e05d_4541_8037_448c0f4cba3e.slice/crio-d38af35cda7dd8cd8bfc859f80871a8ac5a698fa483bb87740da57f6b0230e5d WatchSource:0}: Error finding container d38af35cda7dd8cd8bfc859f80871a8ac5a698fa483bb87740da57f6b0230e5d: Status 404 returned error can't find the container with id d38af35cda7dd8cd8bfc859f80871a8ac5a698fa483bb87740da57f6b0230e5d
Sep 11 17:40:05.934287 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.934251    2582 patch_prober.go:28] interesting pod/image-registry-59bbd88f7d-bkxbf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 17:40:05.934471 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:05.934307    2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf" podUID="20351bca-486b-456e-83b3-8be534dcbf3a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:40:06.089894 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:06.089803    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerStarted","Data":"d38af35cda7dd8cd8bfc859f80871a8ac5a698fa483bb87740da57f6b0230e5d"}
Sep 11 17:40:07.097467 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:07.097430    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx" event={"ID":"9a68b63b-ee60-4ddb-85b9-7129ddd885dc","Type":"ContainerStarted","Data":"55ccfe05b9a6acca8f5410f653c932cb084f755c48f3de74c3c15f4df83709dc"}
Sep 11 17:40:07.097854 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:07.097496    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx" event={"ID":"9a68b63b-ee60-4ddb-85b9-7129ddd885dc","Type":"ContainerStarted","Data":"f00a9e65701cbbc57dea7a4b421d13559544a0eb6a2be4438017913e13734605"}
Sep 11 17:40:07.097854 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:07.097522    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx" event={"ID":"9a68b63b-ee60-4ddb-85b9-7129ddd885dc","Type":"ContainerStarted","Data":"dfa0e3d3a5c1d5dcb12f84103e9ae2337105c1a8e50e013fe4d59e0942837a28"}
Sep 11 17:40:07.994369 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:07.994342    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59bbd88f7d-bkxbf"
Sep 11 17:40:08.103382 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:08.103341    2582 generic.go:356] "Generic (PLEG): container finished" podID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerID="511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917" exitCode=0
Sep 11 17:40:08.103834 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:08.103395    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerDied","Data":"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"}
Sep 11 17:40:09.110875 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:09.110830    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx" event={"ID":"9a68b63b-ee60-4ddb-85b9-7129ddd885dc","Type":"ContainerStarted","Data":"54f2ae4fdb040aafa1930054b1e055cb4761a35f39c87ae2245eeacdc9f721d8"}
Sep 11 17:40:09.111306 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:09.110922    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx" event={"ID":"9a68b63b-ee60-4ddb-85b9-7129ddd885dc","Type":"ContainerStarted","Data":"59a924aa172b24c08a12aa89d24e624e26f90fd795e456a079a19765c15e4eba"}
Sep 11 17:40:09.111306 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:09.110965    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx" event={"ID":"9a68b63b-ee60-4ddb-85b9-7129ddd885dc","Type":"ContainerStarted","Data":"4b290490404da7a36950a41637c5615d13cee3c1b83cb20f01a64a55e59a9d76"}
Sep 11 17:40:09.111306 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:09.111012    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:09.139715 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:09.139659    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx" podStartSLOduration=3.520534337 podStartE2EDuration="8.139605104s" totalImagesPullingTime="4.619070767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:40:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:40:09.137711791 +0000 UTC m=+77.025890138" watchObservedRunningTime="2026-09-11 17:40:09.139605104 +0000 UTC m=+77.027783487"
Sep 11 17:40:12.127205 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:12.127177    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerStarted","Data":"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"}
Sep 11 17:40:12.127487 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:12.127274    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerStarted","Data":"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"}
Sep 11 17:40:12.127487 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:12.127303    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerStarted","Data":"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"}
Sep 11 17:40:12.127487 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:12.127328    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerStarted","Data":"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"}
Sep 11 17:40:13.134186 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:13.134152    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerStarted","Data":"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"}
Sep 11 17:40:13.134566 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:13.134215    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerStarted","Data":"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"}
Sep 11 17:40:13.177256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:13.174578    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.7312612830000003 podStartE2EDuration="8.174556531s" totalImagesPullingTime="5.424622s" totalInitContainerRuntime="18.673248ms" isStatefulPod=false podCreationTimestamp="2026-09-11 17:40:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:40:13.17280035 +0000 UTC m=+81.060978713" watchObservedRunningTime="2026-09-11 17:40:13.174556531 +0000 UTC m=+81.062734877"
Sep 11 17:40:15.121959 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:15.121844    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d7fc48577-vzxsx"
Sep 11 17:40:15.488750 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:15.488714    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:40:24.326170 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:24.326134    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f57fb7d77-z4q5r"]
Sep 11 17:40:35.087341 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:35.087309    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2bq7k"
Sep 11 17:40:49.346071 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.346006    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f57fb7d77-z4q5r" podUID="e5d41f55-083c-493b-ac64-8c23aae3bfd0" containerName="console" containerID="cri-o://57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54" gracePeriod=15
Sep 11 17:40:49.595731 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.595705    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:40:49.627880 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.627794    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-oauth-serving-cert\") pod \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") "
Sep 11 17:40:49.628249 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.628217    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e5d41f55-083c-493b-ac64-8c23aae3bfd0" (UID: "e5d41f55-083c-493b-ac64-8c23aae3bfd0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:40:49.728798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.728745    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-service-ca\") pod \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") "
Sep 11 17:40:49.728798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.728807    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-config\") pod \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") "
Sep 11 17:40:49.729058 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.728834    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2x6s\" (UniqueName: \"kubernetes.io/projected/e5d41f55-083c-493b-ac64-8c23aae3bfd0-kube-api-access-m2x6s\") pod \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") "
Sep 11 17:40:49.729058 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.728857    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-oauth-config\") pod \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") "
Sep 11 17:40:49.729058 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.728881    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-serving-cert\") pod \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\" (UID: \"e5d41f55-083c-493b-ac64-8c23aae3bfd0\") "
Sep 11 17:40:49.729205 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.729104    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-oauth-serving-cert\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:40:49.729257 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.729233    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-service-ca" (OuterVolumeSpecName: "service-ca") pod "e5d41f55-083c-493b-ac64-8c23aae3bfd0" (UID: "e5d41f55-083c-493b-ac64-8c23aae3bfd0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:40:49.729321 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.729294    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-config" (OuterVolumeSpecName: "console-config") pod "e5d41f55-083c-493b-ac64-8c23aae3bfd0" (UID: "e5d41f55-083c-493b-ac64-8c23aae3bfd0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:40:49.731364 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.731332    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e5d41f55-083c-493b-ac64-8c23aae3bfd0" (UID: "e5d41f55-083c-493b-ac64-8c23aae3bfd0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:40:49.731364 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.731358    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e5d41f55-083c-493b-ac64-8c23aae3bfd0" (UID: "e5d41f55-083c-493b-ac64-8c23aae3bfd0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:40:49.731513 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.731428    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5d41f55-083c-493b-ac64-8c23aae3bfd0-kube-api-access-m2x6s" (OuterVolumeSpecName: "kube-api-access-m2x6s") pod "e5d41f55-083c-493b-ac64-8c23aae3bfd0" (UID: "e5d41f55-083c-493b-ac64-8c23aae3bfd0"). InnerVolumeSpecName "kube-api-access-m2x6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:40:49.829926 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.829877    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-service-ca\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:40:49.829926 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.829919    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-config\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:40:49.829926 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.829929    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2x6s\" (UniqueName: \"kubernetes.io/projected/e5d41f55-083c-493b-ac64-8c23aae3bfd0-kube-api-access-m2x6s\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:40:49.829926 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.829937    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-oauth-config\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:40:49.829926 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:49.829946    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5d41f55-083c-493b-ac64-8c23aae3bfd0-console-serving-cert\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:40:50.267199 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:50.267166    2582 generic.go:356] "Generic (PLEG): container finished" podID="e5d41f55-083c-493b-ac64-8c23aae3bfd0" containerID="57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54" exitCode=0
Sep 11 17:40:50.267367 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:50.267205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f57fb7d77-z4q5r" event={"ID":"e5d41f55-083c-493b-ac64-8c23aae3bfd0","Type":"ContainerDied","Data":"57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54"}
Sep 11 17:40:50.267367 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:50.267211    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f57fb7d77-z4q5r"
Sep 11 17:40:50.267367 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:50.267242    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f57fb7d77-z4q5r" event={"ID":"e5d41f55-083c-493b-ac64-8c23aae3bfd0","Type":"ContainerDied","Data":"ac101495d1c04737d4934db9a428cfe217d1a498b84694a39fc1dd47d6db4120"}
Sep 11 17:40:50.267367 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:50.267261    2582 scope.go:117] "RemoveContainer" containerID="57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54"
Sep 11 17:40:50.275714 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:50.275695    2582 scope.go:117] "RemoveContainer" containerID="57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54"
Sep 11 17:40:50.275977 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:40:50.275956    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54\": container with ID starting with 57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54 not found: ID does not exist" containerID="57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54"
Sep 11 17:40:50.276023 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:50.275986    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54"} err="failed to get container status \"57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54\": rpc error: code = NotFound desc = could not find container \"57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54\": container with ID starting with 57a88b9e4301ec003b654e97750df16c129e7f1bc1c5e7686dcf5f6077b29a54 not found: ID does not exist"
Sep 11 17:40:50.290496 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:50.290471    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f57fb7d77-z4q5r"]
Sep 11 17:40:50.294744 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:50.294717    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f57fb7d77-z4q5r"]
Sep 11 17:40:50.632099 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:40:50.632014    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5d41f55-083c-493b-ac64-8c23aae3bfd0" path="/var/lib/kubelet/pods/e5d41f55-083c-493b-ac64-8c23aae3bfd0/volumes"
Sep 11 17:41:05.488984 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:05.488945    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:05.508627 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:05.508600    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:06.341613 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:06.341586    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:23.515806 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.515772    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 17:41:23.516363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.516109    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="prometheus" containerID="cri-o://15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68" gracePeriod=600
Sep 11 17:41:23.516363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.516129    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy" containerID="cri-o://ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc" gracePeriod=600
Sep 11 17:41:23.516363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.516188    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="config-reloader" containerID="cri-o://8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f" gracePeriod=600
Sep 11 17:41:23.516363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.516205    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b" gracePeriod=600
Sep 11 17:41:23.516363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.516221    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy-web" containerID="cri-o://e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002" gracePeriod=600
Sep 11 17:41:23.516363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.516144    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="thanos-sidecar" containerID="cri-o://c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c" gracePeriod=600
Sep 11 17:41:23.761831 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.761804    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:23.862716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.862619    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.862716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.862654    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-1\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.862716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.862674    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-web-config\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.862716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.862691    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w445\" (UniqueName: \"kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-kube-api-access-6w445\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863043 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.862722    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-2\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863043 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.862757    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-0\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863043 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.862789    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-grpc-tls\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863043 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.862816    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863063    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863180    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-kube-rbac-proxy\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863191    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863216    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-tls-assets\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863246    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-db\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863273    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-kubelet-serving-ca-bundle\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863302    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-metrics-client-certs\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863333    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-tls\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863361    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config-out\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863385    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-thanos-prometheus-http-client-file\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863417    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-serving-certs-ca-bundle\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863441    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863472    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-trusted-ca-bundle\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.863586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863502    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-metrics-client-ca\") pod \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\" (UID: \"ac9b48d5-e05d-4541-8037-448c0f4cba3e\") "
Sep 11 17:41:23.864286 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863728    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.864286 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.863744    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.864286 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.864066    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:41:23.864498 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.864469    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:41:23.865646 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.865525    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:41:23.865646 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.865591    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:41:23.866203 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.866061    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:41:23.866540 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.866399    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:41:23.866540 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.866487    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:41:23.866540 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.866483    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:41:23.866766 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.866589    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-kube-api-access-6w445" (OuterVolumeSpecName: "kube-api-access-6w445") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "kube-api-access-6w445". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:41:23.866919 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.866892    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:41:23.867204 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.867152    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:41:23.867541 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.867521    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config-out" (OuterVolumeSpecName: "config-out") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:41:23.867643 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.867590    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config" (OuterVolumeSpecName: "config") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:41:23.868543 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.868522    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:41:23.868616 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.868562    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:41:23.868854 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.868829    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:41:23.868854 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.868844    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:41:23.876757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.876728    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-web-config" (OuterVolumeSpecName: "web-config") pod "ac9b48d5-e05d-4541-8037-448c0f4cba3e" (UID: "ac9b48d5-e05d-4541-8037-448c0f4cba3e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:41:23.964395 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964344    2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-web-config\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964395 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964388    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w445\" (UniqueName: \"kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-kube-api-access-6w445\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964395 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964399    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964395 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964409    2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-grpc-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964420    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964429    2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-kube-rbac-proxy\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964436    2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ac9b48d5-e05d-4541-8037-448c0f4cba3e-tls-assets\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964444    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-k8s-db\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964452    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964461    2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-metrics-client-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964469    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964477    2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config-out\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964485    2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964493    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964501    2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-config\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964508    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964516    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac9b48d5-e05d-4541-8037-448c0f4cba3e-configmap-metrics-client-ca\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:23.964651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:23.964524    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ac9b48d5-e05d-4541-8037-448c0f4cba3e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:41:24.389909 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.389873    2582 generic.go:356] "Generic (PLEG): container finished" podID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerID="7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b" exitCode=0
Sep 11 17:41:24.389909 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.389900    2582 generic.go:356] "Generic (PLEG): container finished" podID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerID="ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc" exitCode=0
Sep 11 17:41:24.389909 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.389909    2582 generic.go:356] "Generic (PLEG): container finished" podID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerID="e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002" exitCode=0
Sep 11 17:41:24.389909 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.389918    2582 generic.go:356] "Generic (PLEG): container finished" podID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerID="c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c" exitCode=0
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.389926    2582 generic.go:356] "Generic (PLEG): container finished" podID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerID="8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f" exitCode=0
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.389933    2582 generic.go:356] "Generic (PLEG): container finished" podID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerID="15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68" exitCode=0
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.389956    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.389956    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerDied","Data":"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"}
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.390030    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerDied","Data":"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"}
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.390058    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerDied","Data":"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"}
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.390104    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerDied","Data":"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"}
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.390134    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerDied","Data":"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"}
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.390158    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerDied","Data":"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"}
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.390108    2582 scope.go:117] "RemoveContainer" containerID="7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"
Sep 11 17:41:24.390252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.390183    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac9b48d5-e05d-4541-8037-448c0f4cba3e","Type":"ContainerDied","Data":"d38af35cda7dd8cd8bfc859f80871a8ac5a698fa483bb87740da57f6b0230e5d"}
Sep 11 17:41:24.398896 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.398874    2582 scope.go:117] "RemoveContainer" containerID="ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"
Sep 11 17:41:24.406527 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.406504    2582 scope.go:117] "RemoveContainer" containerID="e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"
Sep 11 17:41:24.413837 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.413817    2582 scope.go:117] "RemoveContainer" containerID="c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"
Sep 11 17:41:24.415282 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.415052    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 17:41:24.420653 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.420626    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 17:41:24.421974 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.421960    2582 scope.go:117] "RemoveContainer" containerID="8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"
Sep 11 17:41:24.429278 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.429258    2582 scope.go:117] "RemoveContainer" containerID="15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"
Sep 11 17:41:24.438214 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.438186    2582 scope.go:117] "RemoveContainer" containerID="511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"
Sep 11 17:41:24.445660 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.445637    2582 scope.go:117] "RemoveContainer" containerID="7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"
Sep 11 17:41:24.445979 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:41:24.445959    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": container with ID starting with 7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b not found: ID does not exist" containerID="7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"
Sep 11 17:41:24.446025 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.445991    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"} err="failed to get container status \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": rpc error: code = NotFound desc = could not find container \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": container with ID starting with 7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b not found: ID does not exist"
Sep 11 17:41:24.446025 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.446010    2582 scope.go:117] "RemoveContainer" containerID="ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"
Sep 11 17:41:24.446300 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:41:24.446285    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": container with ID starting with ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc not found: ID does not exist" containerID="ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"
Sep 11 17:41:24.446337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.446315    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"} err="failed to get container status \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": rpc error: code = NotFound desc = could not find container \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": container with ID starting with ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc not found: ID does not exist"
Sep 11 17:41:24.446337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.446328    2582 scope.go:117] "RemoveContainer" containerID="e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"
Sep 11 17:41:24.446568 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:41:24.446551    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": container with ID starting with e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002 not found: ID does not exist" containerID="e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"
Sep 11 17:41:24.446626 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.446571    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"} err="failed to get container status \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": rpc error: code = NotFound desc = could not find container \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": container with ID starting with e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002 not found: ID does not exist"
Sep 11 17:41:24.446626 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.446584    2582 scope.go:117] "RemoveContainer" containerID="c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"
Sep 11 17:41:24.446798 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:41:24.446781    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": container with ID starting with c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c not found: ID does not exist" containerID="c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"
Sep 11 17:41:24.446841 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.446807    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"} err="failed to get container status \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": rpc error: code = NotFound desc = could not find container \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": container with ID starting with c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c not found: ID does not exist"
Sep 11 17:41:24.446841 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.446825    2582 scope.go:117] "RemoveContainer" containerID="8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"
Sep 11 17:41:24.447015 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:41:24.446994    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": container with ID starting with 8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f not found: ID does not exist" containerID="8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"
Sep 11 17:41:24.447057 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.447020    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"} err="failed to get container status \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": rpc error: code = NotFound desc = could not find container \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": container with ID starting with 8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f not found: ID does not exist"
Sep 11 17:41:24.447057 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.447035    2582 scope.go:117] "RemoveContainer" containerID="15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"
Sep 11 17:41:24.447374 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:41:24.447352    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": container with ID starting with 15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68 not found: ID does not exist" containerID="15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"
Sep 11 17:41:24.447456 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.447381    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"} err="failed to get container status \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": rpc error: code = NotFound desc = could not find container \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": container with ID starting with 15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68 not found: ID does not exist"
Sep 11 17:41:24.447456 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.447401    2582 scope.go:117] "RemoveContainer" containerID="511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"
Sep 11 17:41:24.447747 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:41:24.447677    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": container with ID starting with 511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917 not found: ID does not exist" containerID="511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"
Sep 11 17:41:24.447808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.447772    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"} err="failed to get container status \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": rpc error: code = NotFound desc = could not find container \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": container with ID starting with 511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917 not found: ID does not exist"
Sep 11 17:41:24.447971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.447944    2582 scope.go:117] "RemoveContainer" containerID="7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"
Sep 11 17:41:24.448328 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.448303    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"} err="failed to get container status \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": rpc error: code = NotFound desc = could not find container \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": container with ID starting with 7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b not found: ID does not exist"
Sep 11 17:41:24.448453 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.448431    2582 scope.go:117] "RemoveContainer" containerID="ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"
Sep 11 17:41:24.448714 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.448693    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"} err="failed to get container status \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": rpc error: code = NotFound desc = could not find container \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": container with ID starting with ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc not found: ID does not exist"
Sep 11 17:41:24.448802 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.448715    2582 scope.go:117] "RemoveContainer" containerID="e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"
Sep 11 17:41:24.448984 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.448965    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"} err="failed to get container status \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": rpc error: code = NotFound desc = could not find container \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": container with ID starting with e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002 not found: ID does not exist"
Sep 11 17:41:24.449035 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.448987    2582 scope.go:117] "RemoveContainer" containerID="c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"
Sep 11 17:41:24.449272 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449237    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"} err="failed to get container status \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": rpc error: code = NotFound desc = could not find container \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": container with ID starting with c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c not found: ID does not exist"
Sep 11 17:41:24.449356 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449274    2582 scope.go:117] "RemoveContainer" containerID="8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"
Sep 11 17:41:24.449488 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449471    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 17:41:24.449579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449562    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"} err="failed to get container status \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": rpc error: code = NotFound desc = could not find container \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": container with ID starting with 8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f not found: ID does not exist"
Sep 11 17:41:24.449648 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449583    2582 scope.go:117] "RemoveContainer" containerID="15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"
Sep 11 17:41:24.449749 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449734    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy-web"
Sep 11 17:41:24.449792 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449753    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy-web"
Sep 11 17:41:24.449792 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449766    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="thanos-sidecar"
Sep 11 17:41:24.449792 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449775    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="thanos-sidecar"
Sep 11 17:41:24.449792 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449783    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy"
Sep 11 17:41:24.449792 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449792    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449809    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="prometheus"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449816    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="prometheus"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449825    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="init-config-reloader"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449830    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="init-config-reloader"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449839    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="config-reloader"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449843    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="config-reloader"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449848    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy-thanos"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449855    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy-thanos"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449784    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"} err="failed to get container status \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": rpc error: code = NotFound desc = could not find container \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": container with ID starting with 15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68 not found: ID does not exist"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449876    2582 scope.go:117] "RemoveContainer" containerID="511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449866    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5d41f55-083c-493b-ac64-8c23aae3bfd0" containerName="console"
Sep 11 17:41:24.449943 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449922    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5d41f55-083c-493b-ac64-8c23aae3bfd0" containerName="console"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449981    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy-thanos"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449988    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5d41f55-083c-493b-ac64-8c23aae3bfd0" containerName="console"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.449996    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="config-reloader"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450003    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450011    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="thanos-sidecar"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450019    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="prometheus"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450024    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" containerName="kube-rbac-proxy-web"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450220    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"} err="failed to get container status \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": rpc error: code = NotFound desc = could not find container \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": container with ID starting with 511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917 not found: ID does not exist"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450246    2582 scope.go:117] "RemoveContainer" containerID="7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450483    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"} err="failed to get container status \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": rpc error: code = NotFound desc = could not find container \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": container with ID starting with 7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b not found: ID does not exist"
Sep 11 17:41:24.450579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450514    2582 scope.go:117] "RemoveContainer" containerID="ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"
Sep 11 17:41:24.450944 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450775    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"} err="failed to get container status \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": rpc error: code = NotFound desc = could not find container \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": container with ID starting with ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc not found: ID does not exist"
Sep 11 17:41:24.450944 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450792    2582 scope.go:117] "RemoveContainer" containerID="e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"
Sep 11 17:41:24.451021 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.450988    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"} err="failed to get container status \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": rpc error: code = NotFound desc = could not find container \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": container with ID starting with e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002 not found: ID does not exist"
Sep 11 17:41:24.451021 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.451004    2582 scope.go:117] "RemoveContainer" containerID="c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"
Sep 11 17:41:24.451237 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.451220    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"} err="failed to get container status \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": rpc error: code = NotFound desc = could not find container \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": container with ID starting with c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c not found: ID does not exist"
Sep 11 17:41:24.451312 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.451238    2582 scope.go:117] "RemoveContainer" containerID="8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"
Sep 11 17:41:24.451473 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.451456    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"} err="failed to get container status \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": rpc error: code = NotFound desc = could not find container \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": container with ID starting with 8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f not found: ID does not exist"
Sep 11 17:41:24.451525 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.451474    2582 scope.go:117] "RemoveContainer" containerID="15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"
Sep 11 17:41:24.451696 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.451679    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"} err="failed to get container status \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": rpc error: code = NotFound desc = could not find container \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": container with ID starting with 15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68 not found: ID does not exist"
Sep 11 17:41:24.451737 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.451698    2582 scope.go:117] "RemoveContainer" containerID="511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"
Sep 11 17:41:24.451907 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.451892    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"} err="failed to get container status \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": rpc error: code = NotFound desc = could not find container \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": container with ID starting with 511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917 not found: ID does not exist"
Sep 11 17:41:24.451944 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.451908    2582 scope.go:117] "RemoveContainer" containerID="7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"
Sep 11 17:41:24.452124 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.452107    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"} err="failed to get container status \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": rpc error: code = NotFound desc = could not find container \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": container with ID starting with 7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b not found: ID does not exist"
Sep 11 17:41:24.452170 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.452124    2582 scope.go:117] "RemoveContainer" containerID="ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"
Sep 11 17:41:24.452312 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.452298    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"} err="failed to get container status \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": rpc error: code = NotFound desc = could not find container \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": container with ID starting with ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc not found: ID does not exist"
Sep 11 17:41:24.452353 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.452312    2582 scope.go:117] "RemoveContainer" containerID="e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"
Sep 11 17:41:24.452477 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.452462    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"} err="failed to get container status \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": rpc error: code = NotFound desc = could not find container \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": container with ID starting with e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002 not found: ID does not exist"
Sep 11 17:41:24.452520 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.452477    2582 scope.go:117] "RemoveContainer" containerID="c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"
Sep 11 17:41:24.452713 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.452690    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"} err="failed to get container status \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": rpc error: code = NotFound desc = could not find container \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": container with ID starting with c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c not found: ID does not exist"
Sep 11 17:41:24.452713 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.452707    2582 scope.go:117] "RemoveContainer" containerID="8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"
Sep 11 17:41:24.452880 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.452864    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"} err="failed to get container status \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": rpc error: code = NotFound desc = could not find container \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": container with ID starting with 8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f not found: ID does not exist"
Sep 11 17:41:24.452929 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.452881    2582 scope.go:117] "RemoveContainer" containerID="15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"
Sep 11 17:41:24.453051 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.453036    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"} err="failed to get container status \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": rpc error: code = NotFound desc = could not find container \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": container with ID starting with 15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68 not found: ID does not exist"
Sep 11 17:41:24.453051 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.453050    2582 scope.go:117] "RemoveContainer" containerID="511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"
Sep 11 17:41:24.453250 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.453234    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"} err="failed to get container status \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": rpc error: code = NotFound desc = could not find container \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": container with ID starting with 511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917 not found: ID does not exist"
Sep 11 17:41:24.453250 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.453248    2582 scope.go:117] "RemoveContainer" containerID="7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"
Sep 11 17:41:24.453438 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.453423    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"} err="failed to get container status \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": rpc error: code = NotFound desc = could not find container \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": container with ID starting with 7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b not found: ID does not exist"
Sep 11 17:41:24.453472 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.453439    2582 scope.go:117] "RemoveContainer" containerID="ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"
Sep 11 17:41:24.453662 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.453645    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"} err="failed to get container status \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": rpc error: code = NotFound desc = could not find container \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": container with ID starting with ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc not found: ID does not exist"
Sep 11 17:41:24.453705 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.453662    2582 scope.go:117] "RemoveContainer" containerID="e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"
Sep 11 17:41:24.453877 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.453862    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"} err="failed to get container status \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": rpc error: code = NotFound desc = could not find container \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": container with ID starting with e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002 not found: ID does not exist"
Sep 11 17:41:24.453877 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.453876    2582 scope.go:117] "RemoveContainer" containerID="c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"
Sep 11 17:41:24.454106 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.454068    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"} err="failed to get container status \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": rpc error: code = NotFound desc = could not find container \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": container with ID starting with c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c not found: ID does not exist"
Sep 11 17:41:24.454147 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.454107    2582 scope.go:117] "RemoveContainer" containerID="8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"
Sep 11 17:41:24.454266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.454252    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"} err="failed to get container status \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": rpc error: code = NotFound desc = could not find container \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": container with ID starting with 8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f not found: ID does not exist"
Sep 11 17:41:24.454312 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.454265    2582 scope.go:117] "RemoveContainer" containerID="15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"
Sep 11 17:41:24.454442 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.454427    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"} err="failed to get container status \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": rpc error: code = NotFound desc = could not find container \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": container with ID starting with 15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68 not found: ID does not exist"
Sep 11 17:41:24.454483 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.454441    2582 scope.go:117] "RemoveContainer" containerID="511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"
Sep 11 17:41:24.454650 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.454631    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"} err="failed to get container status \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": rpc error: code = NotFound desc = could not find container \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": container with ID starting with 511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917 not found: ID does not exist"
Sep 11 17:41:24.454702 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.454650    2582 scope.go:117] "RemoveContainer" containerID="7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"
Sep 11 17:41:24.454851 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.454834    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b"} err="failed to get container status \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": rpc error: code = NotFound desc = could not find container \"7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b\": container with ID starting with 7cc1743f2ce2d6f8685a99cd417c22a6726288fe2ca1efc3c0fa772a9e29ad6b not found: ID does not exist"
Sep 11 17:41:24.454893 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.454851    2582 scope.go:117] "RemoveContainer" containerID="ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"
Sep 11 17:41:24.455047 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.455028    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc"} err="failed to get container status \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": rpc error: code = NotFound desc = could not find container \"ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc\": container with ID starting with ec2bbf1aee9f36961bf672cfea03a741757bdc567b1e5ca31ebcce3fb2db0bdc not found: ID does not exist"
Sep 11 17:41:24.455143 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.455049    2582 scope.go:117] "RemoveContainer" containerID="e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"
Sep 11 17:41:24.455280 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.455265    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002"} err="failed to get container status \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": rpc error: code = NotFound desc = could not find container \"e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002\": container with ID starting with e00050b947a093dad22e8e1afe2f205ebc080e50d543a77465fbbb445ab8f002 not found: ID does not exist"
Sep 11 17:41:24.455322 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.455281    2582 scope.go:117] "RemoveContainer" containerID="c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"
Sep 11 17:41:24.455462 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.455444    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c"} err="failed to get container status \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": rpc error: code = NotFound desc = could not find container \"c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c\": container with ID starting with c9cbb1305ad172450cbc9858268033f831c85b90099f56ab5edb584cfa4cdd8c not found: ID does not exist"
Sep 11 17:41:24.455501 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.455463    2582 scope.go:117] "RemoveContainer" containerID="8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"
Sep 11 17:41:24.455652 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.455633    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f"} err="failed to get container status \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": rpc error: code = NotFound desc = could not find container \"8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f\": container with ID starting with 8e2856da28cb2bac6d3aafdbf50c3a031a885fc4e993830e2db3c1358eec596f not found: ID does not exist"
Sep 11 17:41:24.455713 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.455654    2582 scope.go:117] "RemoveContainer" containerID="15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"
Sep 11 17:41:24.455851 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.455834    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68"} err="failed to get container status \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": rpc error: code = NotFound desc = could not find container \"15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68\": container with ID starting with 15377ebe4a3c14c3543d4429067a692f2a8fcb8cc9be06d5cf51f7d9631a5a68 not found: ID does not exist"
Sep 11 17:41:24.455898 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.455851    2582 scope.go:117] "RemoveContainer" containerID="511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"
Sep 11 17:41:24.456097 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.456061    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917"} err="failed to get container status \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": rpc error: code = NotFound desc = could not find container \"511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917\": container with ID starting with 511d1755df5af5cd6f83a65bd66cbebee810c7028459b3129868614d16682917 not found: ID does not exist"
Sep 11 17:41:24.456697 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.456683    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.459590 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.459568    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 17:41:24.459807 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.459791    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 17:41:24.459869 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.459821    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 17:41:24.459932 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.459876    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 17:41:24.460128 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.460104    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a2eb7dbhgc9vf\""
Sep 11 17:41:24.460211 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.460168    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 17:41:24.460272 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.460218    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 17:41:24.460359 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.460342    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 17:41:24.460415 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.460346    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 17:41:24.460465 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.460458    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 17:41:24.460756 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.460742    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 17:41:24.461050 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.461030    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 17:41:24.461184 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.461073    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 17:41:24.461184 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.461036    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m2mg5\""
Sep 11 17:41:24.461184 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.461113    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 17:41:24.463907 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.463888    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 17:41:24.467766 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.467743    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 17:41:24.476849 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.475845    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 17:41:24.568732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.568682    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-web-config\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.568732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.568730    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c5c7f2-9872-4065-b1cf-f80735431b17-config-out\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.568751    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.568777    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.568834    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.568884    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.568911    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.568943    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c5c7f2-9872-4065-b1cf-f80735431b17-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.568995    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569016    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569035    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569104    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569154    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569202    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569256 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6tqd\" (UniqueName: \"kubernetes.io/projected/28c5c7f2-9872-4065-b1cf-f80735431b17-kube-api-access-z6tqd\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569780 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569291    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-config\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.569780 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.569338    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.632877 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.632838    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac9b48d5-e05d-4541-8037-448c0f4cba3e" path="/var/lib/kubelet/pods/ac9b48d5-e05d-4541-8037-448c0f4cba3e/volumes"
Sep 11 17:41:24.670223 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670192    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670399 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670229    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670399 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670255    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670399 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670281    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670399 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670314    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670399 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670344    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670399 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670365    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670399 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670394    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670439    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670463    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6tqd\" (UniqueName: \"kubernetes.io/projected/28c5c7f2-9872-4065-b1cf-f80735431b17-kube-api-access-z6tqd\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-config\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670530    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670563    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-web-config\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c5c7f2-9872-4065-b1cf-f80735431b17-config-out\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670613    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670678    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670721    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.670751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670747    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.671327 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c5c7f2-9872-4065-b1cf-f80735431b17-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.671327 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.670919    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.671327 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.671017    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.672175 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.671822    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.672175 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.671911    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.672478 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.672452    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.673920 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.673605    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.673920 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.673767    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c5c7f2-9872-4065-b1cf-f80735431b17-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.674096 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.674029    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.674515 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.674239    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-config\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.674616 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.674547    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.675225 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.674894    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.675225 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.674968    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5c7f2-9872-4065-b1cf-f80735431b17-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.675225 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.675180    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.675417 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.675264    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.675669 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.675645    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.676284 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.676259    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-web-config\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.676531 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.676514    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c5c7f2-9872-4065-b1cf-f80735431b17-config-out\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.676973 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.676954    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.677849 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.677830    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c5c7f2-9872-4065-b1cf-f80735431b17-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.685255 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.685233    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6tqd\" (UniqueName: \"kubernetes.io/projected/28c5c7f2-9872-4065-b1cf-f80735431b17-kube-api-access-z6tqd\") pod \"prometheus-k8s-0\" (UID: \"28c5c7f2-9872-4065-b1cf-f80735431b17\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:24.766245 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:24.766211    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:41:25.047894 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:25.047868    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 17:41:25.049494 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:41:25.049467    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c5c7f2_9872_4065_b1cf_f80735431b17.slice/crio-d510ebcfa4a6077748963eeb4e3829f2f79b179eb0807c3545f9a0714e993ab5 WatchSource:0}: Error finding container d510ebcfa4a6077748963eeb4e3829f2f79b179eb0807c3545f9a0714e993ab5: Status 404 returned error can't find the container with id d510ebcfa4a6077748963eeb4e3829f2f79b179eb0807c3545f9a0714e993ab5
Sep 11 17:41:25.395619 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:25.395589    2582 generic.go:356] "Generic (PLEG): container finished" podID="28c5c7f2-9872-4065-b1cf-f80735431b17" containerID="3c96991aab440063aaf2ffddcdf830b87d0e6451a42a8f775e91231ede7626a2" exitCode=0
Sep 11 17:41:25.395797 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:25.395665    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28c5c7f2-9872-4065-b1cf-f80735431b17","Type":"ContainerDied","Data":"3c96991aab440063aaf2ffddcdf830b87d0e6451a42a8f775e91231ede7626a2"}
Sep 11 17:41:25.395797 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:25.395723    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28c5c7f2-9872-4065-b1cf-f80735431b17","Type":"ContainerStarted","Data":"d510ebcfa4a6077748963eeb4e3829f2f79b179eb0807c3545f9a0714e993ab5"}
Sep 11 17:41:26.403504 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:26.403462    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28c5c7f2-9872-4065-b1cf-f80735431b17","Type":"ContainerStarted","Data":"fb0d3d3534eb3d599174729ed6126ece25f40ca0241d0d3ba7ad4dc4d2e36c69"}
Sep 11 17:41:26.403944 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:26.403554    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28c5c7f2-9872-4065-b1cf-f80735431b17","Type":"ContainerStarted","Data":"c36301446a020ed2c5d6bac09c49fb4260c80916cf9ab49f1c244a27a4a99650"}
Sep 11 17:41:26.403944 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:26.403580    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28c5c7f2-9872-4065-b1cf-f80735431b17","Type":"ContainerStarted","Data":"b963951135acc44d779baff61f49c3d322e6195a83626af6279537faf0dcb7d5"}
Sep 11 17:41:26.403944 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:26.403604    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28c5c7f2-9872-4065-b1cf-f80735431b17","Type":"ContainerStarted","Data":"5618bcd10f1851c5df20587a26ce89a19763b46e51645131a9ca1c4536e3cd55"}
Sep 11 17:41:26.403944 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:26.403641    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28c5c7f2-9872-4065-b1cf-f80735431b17","Type":"ContainerStarted","Data":"311e89cb1479e44fb25de7a8945387a9730778042be31c7cf6ebc43e639caa23"}
Sep 11 17:41:26.403944 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:26.403669    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28c5c7f2-9872-4065-b1cf-f80735431b17","Type":"ContainerStarted","Data":"ef24c2dbb255b4dd51a34b774efa2d251ef0af8a24d9a1db1dbe022d5a02665e"}
Sep 11 17:41:26.436467 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:26.436416    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.4185212480000002 podStartE2EDuration="2.436397539s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.876291ms" isStatefulPod=false podCreationTimestamp="2026-09-11 17:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:41:26.43431218 +0000 UTC m=+154.322490526" watchObservedRunningTime="2026-09-11 17:41:26.436397539 +0000 UTC m=+154.324575889"
Sep 11 17:41:29.767370 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:41:29.767328    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:42:24.767175 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:42:24.767135    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:42:24.782651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:42:24.782620    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:42:25.630193 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:42:25.630166    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 17:43:19.757464 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.757386    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"]
Sep 11 17:43:19.760470 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.760450    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"
Sep 11 17:43:19.763369 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.763339    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:43:19.763461 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.763392    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-xz9gh\""
Sep 11 17:43:19.764697 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.764682    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 17:43:19.769823 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.769799    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"]
Sep 11 17:43:19.863429 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.863395    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn9jl\" (UniqueName: \"kubernetes.io/projected/d8cef1c4-4e1e-4d7b-b655-1cdcd495982f-kube-api-access-cn9jl\") pod \"openshift-lws-operator-fd8ccff4c-llpns\" (UID: \"d8cef1c4-4e1e-4d7b-b655-1cdcd495982f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"
Sep 11 17:43:19.863609 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.863455    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8cef1c4-4e1e-4d7b-b655-1cdcd495982f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-llpns\" (UID: \"d8cef1c4-4e1e-4d7b-b655-1cdcd495982f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"
Sep 11 17:43:19.964821 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.964791    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn9jl\" (UniqueName: \"kubernetes.io/projected/d8cef1c4-4e1e-4d7b-b655-1cdcd495982f-kube-api-access-cn9jl\") pod \"openshift-lws-operator-fd8ccff4c-llpns\" (UID: \"d8cef1c4-4e1e-4d7b-b655-1cdcd495982f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"
Sep 11 17:43:19.964939 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.964847    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8cef1c4-4e1e-4d7b-b655-1cdcd495982f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-llpns\" (UID: \"d8cef1c4-4e1e-4d7b-b655-1cdcd495982f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"
Sep 11 17:43:19.965232 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.965212    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8cef1c4-4e1e-4d7b-b655-1cdcd495982f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-llpns\" (UID: \"d8cef1c4-4e1e-4d7b-b655-1cdcd495982f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"
Sep 11 17:43:19.975266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:19.975244    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn9jl\" (UniqueName: \"kubernetes.io/projected/d8cef1c4-4e1e-4d7b-b655-1cdcd495982f-kube-api-access-cn9jl\") pod \"openshift-lws-operator-fd8ccff4c-llpns\" (UID: \"d8cef1c4-4e1e-4d7b-b655-1cdcd495982f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"
Sep 11 17:43:20.077924 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:20.077833    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"
Sep 11 17:43:20.348827 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:20.348734    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns"]
Sep 11 17:43:20.351832 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:43:20.351804    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8cef1c4_4e1e_4d7b_b655_1cdcd495982f.slice/crio-cbbfee9b411c0ab6c25dda9005a0681c8cabdec09170e0054d1f36788de11672 WatchSource:0}: Error finding container cbbfee9b411c0ab6c25dda9005a0681c8cabdec09170e0054d1f36788de11672: Status 404 returned error can't find the container with id cbbfee9b411c0ab6c25dda9005a0681c8cabdec09170e0054d1f36788de11672
Sep 11 17:43:20.804839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:20.804801    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns" event={"ID":"d8cef1c4-4e1e-4d7b-b655-1cdcd495982f","Type":"ContainerStarted","Data":"cbbfee9b411c0ab6c25dda9005a0681c8cabdec09170e0054d1f36788de11672"}
Sep 11 17:43:22.814450 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:22.814407    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns" event={"ID":"d8cef1c4-4e1e-4d7b-b655-1cdcd495982f","Type":"ContainerStarted","Data":"26feda8ef7ef667b2350148c1a15e4188bdf539d6c1b30c1615c14d07cbd590b"}
Sep 11 17:43:22.832233 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:22.832178    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-llpns" podStartSLOduration=1.491814921 podStartE2EDuration="3.832159321s" totalImagesPullingTime="2.3403444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:43:22.831923741 +0000 UTC m=+270.720102088" watchObservedRunningTime="2026-09-11 17:43:22.832159321 +0000 UTC m=+270.720337684"
Sep 11 17:43:40.636656 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.636623    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"]
Sep 11 17:43:40.644998 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.644968    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.650160 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.650134    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 17:43:40.650373 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.650350    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 17:43:40.650611 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.650430    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 17:43:40.650611 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.650481    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jvzqs\""
Sep 11 17:43:40.652010 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.651991    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 17:43:40.681660 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.681623    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"]
Sep 11 17:43:40.719842 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.719800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42730934-e078-40e1-b99f-5eddcabf1653-tmp\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.720033 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.719884    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42730934-e078-40e1-b99f-5eddcabf1653-webhook-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.720033 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.719938    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgww2\" (UniqueName: \"kubernetes.io/projected/42730934-e078-40e1-b99f-5eddcabf1653-kube-api-access-sgww2\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.720033 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.720001    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/42730934-e078-40e1-b99f-5eddcabf1653-manifests\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.720187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.720036    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/42730934-e078-40e1-b99f-5eddcabf1653-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.820420 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.820385    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42730934-e078-40e1-b99f-5eddcabf1653-webhook-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.820420 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.820420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgww2\" (UniqueName: \"kubernetes.io/projected/42730934-e078-40e1-b99f-5eddcabf1653-kube-api-access-sgww2\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.820705 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.820472    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/42730934-e078-40e1-b99f-5eddcabf1653-manifests\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.820705 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.820492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/42730934-e078-40e1-b99f-5eddcabf1653-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.820705 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.820509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42730934-e078-40e1-b99f-5eddcabf1653-tmp\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.820990 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.820959    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/42730934-e078-40e1-b99f-5eddcabf1653-manifests\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.822983 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.822960    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42730934-e078-40e1-b99f-5eddcabf1653-tmp\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.823125 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.823107    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/42730934-e078-40e1-b99f-5eddcabf1653-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.823365 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.823346    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42730934-e078-40e1-b99f-5eddcabf1653-webhook-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.830265 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.830237    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgww2\" (UniqueName: \"kubernetes.io/projected/42730934-e078-40e1-b99f-5eddcabf1653-kube-api-access-sgww2\") pod \"opendatahub-operator-controller-manager-86997d8dbf-58pml\" (UID: \"42730934-e078-40e1-b99f-5eddcabf1653\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:40.955695 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:40.955650    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:41.236424 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:41.236398    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"]
Sep 11 17:43:41.238367 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:43:41.238340    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42730934_e078_40e1_b99f_5eddcabf1653.slice/crio-977d7347f9bb0cbb64d65d5b2f977a2211f87237b109488b614ff3ed423767b1 WatchSource:0}: Error finding container 977d7347f9bb0cbb64d65d5b2f977a2211f87237b109488b614ff3ed423767b1: Status 404 returned error can't find the container with id 977d7347f9bb0cbb64d65d5b2f977a2211f87237b109488b614ff3ed423767b1
Sep 11 17:43:41.887040 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:41.886993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml" event={"ID":"42730934-e078-40e1-b99f-5eddcabf1653","Type":"ContainerStarted","Data":"977d7347f9bb0cbb64d65d5b2f977a2211f87237b109488b614ff3ed423767b1"}
Sep 11 17:43:44.899688 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:44.899652    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml" event={"ID":"42730934-e078-40e1-b99f-5eddcabf1653","Type":"ContainerStarted","Data":"94368fe0465510c8945249198279ea473a2bb94e2da46de962e32fa306945947"}
Sep 11 17:43:45.905219 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:45.905179    2582 generic.go:356] "Generic (PLEG): container finished" podID="42730934-e078-40e1-b99f-5eddcabf1653" containerID="94368fe0465510c8945249198279ea473a2bb94e2da46de962e32fa306945947" exitCode=0
Sep 11 17:43:45.905737 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:45.905243    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml" event={"ID":"42730934-e078-40e1-b99f-5eddcabf1653","Type":"ContainerDied","Data":"94368fe0465510c8945249198279ea473a2bb94e2da46de962e32fa306945947"}
Sep 11 17:43:46.910376 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:46.910335    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml" event={"ID":"42730934-e078-40e1-b99f-5eddcabf1653","Type":"ContainerStarted","Data":"2b62df0cfe47f09cbbff15c4715795f9433f94615b6e9547ebb05ef7f71f4fd6"}
Sep 11 17:43:46.910855 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:46.910450    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:46.933657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:46.933597    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml" podStartSLOduration=2.594528019 podStartE2EDuration="6.933571987s" totalImagesPullingTime="3.306028235s" totalInitContainerRuntime="1.033015733s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:43:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:43:46.932073639 +0000 UTC m=+294.820251995" watchObservedRunningTime="2026-09-11 17:43:46.933571987 +0000 UTC m=+294.821750334"
Sep 11 17:43:52.510027 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:52.510001    2582 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 17:43:57.917271 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:57.917233    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-58pml"
Sep 11 17:43:59.644017 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.643984    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"]
Sep 11 17:43:59.646138 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.646121    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.653109 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.653065    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 17:43:59.653218 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.653201    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 17:43:59.653803 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.653769    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 17:43:59.656738 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.656722    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kkfd2\""
Sep 11 17:43:59.684648 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.684608    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"]
Sep 11 17:43:59.771496 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.771463    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8d7996b-d427-4bc2-911b-6f72ef3a8921-metrics-cert\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.771496 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.771497    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp2dh\" (UniqueName: \"kubernetes.io/projected/d8d7996b-d427-4bc2-911b-6f72ef3a8921-kube-api-access-lp2dh\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.771711 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.771521    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d8d7996b-d427-4bc2-911b-6f72ef3a8921-manager-config\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.771711 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.771561    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8d7996b-d427-4bc2-911b-6f72ef3a8921-cert\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.873056 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.873019    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8d7996b-d427-4bc2-911b-6f72ef3a8921-metrics-cert\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.873056 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.873066    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp2dh\" (UniqueName: \"kubernetes.io/projected/d8d7996b-d427-4bc2-911b-6f72ef3a8921-kube-api-access-lp2dh\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.873422 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.873229    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d8d7996b-d427-4bc2-911b-6f72ef3a8921-manager-config\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.873422 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.873338    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8d7996b-d427-4bc2-911b-6f72ef3a8921-cert\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.873818 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.873792    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d8d7996b-d427-4bc2-911b-6f72ef3a8921-manager-config\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.875759 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.875733    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8d7996b-d427-4bc2-911b-6f72ef3a8921-cert\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.875859 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.875794    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8d7996b-d427-4bc2-911b-6f72ef3a8921-metrics-cert\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.903070 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.903002    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp2dh\" (UniqueName: \"kubernetes.io/projected/d8d7996b-d427-4bc2-911b-6f72ef3a8921-kube-api-access-lp2dh\") pod \"lws-controller-manager-6476dc957d-sbsnk\" (UID: \"d8d7996b-d427-4bc2-911b-6f72ef3a8921\") " pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:43:59.956056 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:43:59.956014    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:44:00.270557 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:00.270525    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"]
Sep 11 17:44:00.271282 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:44:00.271255    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8d7996b_d427_4bc2_911b_6f72ef3a8921.slice/crio-0c815b5504f204135ba25aeaca882864ec30a97ed352e413d0de43d5fabf8e1b WatchSource:0}: Error finding container 0c815b5504f204135ba25aeaca882864ec30a97ed352e413d0de43d5fabf8e1b: Status 404 returned error can't find the container with id 0c815b5504f204135ba25aeaca882864ec30a97ed352e413d0de43d5fabf8e1b
Sep 11 17:44:00.273222 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:00.273201    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:44:00.965991 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:00.965949    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk" event={"ID":"d8d7996b-d427-4bc2-911b-6f72ef3a8921","Type":"ContainerStarted","Data":"0c815b5504f204135ba25aeaca882864ec30a97ed352e413d0de43d5fabf8e1b"}
Sep 11 17:44:01.970940 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:01.970839    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk" event={"ID":"d8d7996b-d427-4bc2-911b-6f72ef3a8921","Type":"ContainerStarted","Data":"1e7f3b380f5e34e39f04454e4eda462dac6a887b9ba6f986d0b66eb8557e7ea8"}
Sep 11 17:44:01.971425 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:01.970968    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:44:01.991680 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:01.991625    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk" podStartSLOduration=1.559407617 podStartE2EDuration="2.991583556s" totalImagesPullingTime="1.432175939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:44:01.989954412 +0000 UTC m=+309.878132759" watchObservedRunningTime="2026-09-11 17:44:01.991583556 +0000 UTC m=+309.879761905"
Sep 11 17:44:05.732715 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.732680    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"]
Sep 11 17:44:05.735109 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.735055    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:05.738324 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.738292    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 17:44:05.741665 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.741634    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2dqgf\""
Sep 11 17:44:05.741781 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.741682    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 17:44:05.742870 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.742853    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 17:44:05.742949 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.742864    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 17:44:05.747494 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.747473    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"]
Sep 11 17:44:05.823856 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.823812    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae939b3e-3286-44d3-88dd-6f67a1f79dc6-tmp\") pod \"kube-auth-proxy-64d8557cb6-c5r7j\" (UID: \"ae939b3e-3286-44d3-88dd-6f67a1f79dc6\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:05.824038 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.823873    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb74r\" (UniqueName: \"kubernetes.io/projected/ae939b3e-3286-44d3-88dd-6f67a1f79dc6-kube-api-access-pb74r\") pod \"kube-auth-proxy-64d8557cb6-c5r7j\" (UID: \"ae939b3e-3286-44d3-88dd-6f67a1f79dc6\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:05.824038 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.823926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae939b3e-3286-44d3-88dd-6f67a1f79dc6-tls-certs\") pod \"kube-auth-proxy-64d8557cb6-c5r7j\" (UID: \"ae939b3e-3286-44d3-88dd-6f67a1f79dc6\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:05.924438 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.924398    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae939b3e-3286-44d3-88dd-6f67a1f79dc6-tmp\") pod \"kube-auth-proxy-64d8557cb6-c5r7j\" (UID: \"ae939b3e-3286-44d3-88dd-6f67a1f79dc6\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:05.924616 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.924459    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb74r\" (UniqueName: \"kubernetes.io/projected/ae939b3e-3286-44d3-88dd-6f67a1f79dc6-kube-api-access-pb74r\") pod \"kube-auth-proxy-64d8557cb6-c5r7j\" (UID: \"ae939b3e-3286-44d3-88dd-6f67a1f79dc6\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:05.924616 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.924484    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae939b3e-3286-44d3-88dd-6f67a1f79dc6-tls-certs\") pod \"kube-auth-proxy-64d8557cb6-c5r7j\" (UID: \"ae939b3e-3286-44d3-88dd-6f67a1f79dc6\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:05.926876 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.926849    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae939b3e-3286-44d3-88dd-6f67a1f79dc6-tmp\") pod \"kube-auth-proxy-64d8557cb6-c5r7j\" (UID: \"ae939b3e-3286-44d3-88dd-6f67a1f79dc6\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:05.927042 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.927026    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae939b3e-3286-44d3-88dd-6f67a1f79dc6-tls-certs\") pod \"kube-auth-proxy-64d8557cb6-c5r7j\" (UID: \"ae939b3e-3286-44d3-88dd-6f67a1f79dc6\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:05.933537 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:05.933510    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb74r\" (UniqueName: \"kubernetes.io/projected/ae939b3e-3286-44d3-88dd-6f67a1f79dc6-kube-api-access-pb74r\") pod \"kube-auth-proxy-64d8557cb6-c5r7j\" (UID: \"ae939b3e-3286-44d3-88dd-6f67a1f79dc6\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:06.049844 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:06.049756    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"
Sep 11 17:44:06.324308 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:06.324277    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j"]
Sep 11 17:44:06.326311 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:44:06.326284    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae939b3e_3286_44d3_88dd_6f67a1f79dc6.slice/crio-fae0c2e997cb1184da27c8b54285a58e93b8a08121dc0718accd68afe9ab8896 WatchSource:0}: Error finding container fae0c2e997cb1184da27c8b54285a58e93b8a08121dc0718accd68afe9ab8896: Status 404 returned error can't find the container with id fae0c2e997cb1184da27c8b54285a58e93b8a08121dc0718accd68afe9ab8896
Sep 11 17:44:06.991325 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:06.991288    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j" event={"ID":"ae939b3e-3286-44d3-88dd-6f67a1f79dc6","Type":"ContainerStarted","Data":"fae0c2e997cb1184da27c8b54285a58e93b8a08121dc0718accd68afe9ab8896"}
Sep 11 17:44:10.006290 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:10.006245    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j" event={"ID":"ae939b3e-3286-44d3-88dd-6f67a1f79dc6","Type":"ContainerStarted","Data":"f1e751e373aa5a8ac30495d2bb242455b4c0236b712f87511d238e260a4a2a20"}
Sep 11 17:44:10.026944 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:10.026887    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64d8557cb6-c5r7j" podStartSLOduration=2.150183425 podStartE2EDuration="5.026868911s" totalImagesPullingTime="2.876685486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:44:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:44:10.025704019 +0000 UTC m=+317.913882366" watchObservedRunningTime="2026-09-11 17:44:10.026868911 +0000 UTC m=+317.915047256"
Sep 11 17:44:12.977004 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:12.976959    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6476dc957d-sbsnk"
Sep 11 17:44:21.703595 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:21.703558    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"]
Sep 11 17:44:21.707027 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:21.706999    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:44:21.710307 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:21.710277    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 11 17:44:21.710448 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:21.710371    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-whrqj\""
Sep 11 17:44:21.721358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:21.721323    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"]
Sep 11 17:44:21.862246 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:21.862206    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7818d3c-4200-40bd-a528-f4266df97057-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nfbc7\" (UID: \"d7818d3c-4200-40bd-a528-f4266df97057\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:44:21.862246 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:21.862258    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46n2s\" (UniqueName: \"kubernetes.io/projected/d7818d3c-4200-40bd-a528-f4266df97057-kube-api-access-46n2s\") pod \"llmisvc-controller-manager-777c6bd7d5-nfbc7\" (UID: \"d7818d3c-4200-40bd-a528-f4266df97057\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:44:21.963490 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:21.963397    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7818d3c-4200-40bd-a528-f4266df97057-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nfbc7\" (UID: \"d7818d3c-4200-40bd-a528-f4266df97057\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:44:21.963490 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:21.963444    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46n2s\" (UniqueName: \"kubernetes.io/projected/d7818d3c-4200-40bd-a528-f4266df97057-kube-api-access-46n2s\") pod \"llmisvc-controller-manager-777c6bd7d5-nfbc7\" (UID: \"d7818d3c-4200-40bd-a528-f4266df97057\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:44:21.963685 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:44:21.963602    2582 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 11 17:44:21.963685 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:44:21.963679    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7818d3c-4200-40bd-a528-f4266df97057-cert podName:d7818d3c-4200-40bd-a528-f4266df97057 nodeName:}" failed. No retries permitted until 2026-09-11 17:44:22.463660192 +0000 UTC m=+330.351838536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d7818d3c-4200-40bd-a528-f4266df97057-cert") pod "llmisvc-controller-manager-777c6bd7d5-nfbc7" (UID: "d7818d3c-4200-40bd-a528-f4266df97057") : secret "llmisvc-webhook-server-cert" not found
Sep 11 17:44:21.980482 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:21.980406    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46n2s\" (UniqueName: \"kubernetes.io/projected/d7818d3c-4200-40bd-a528-f4266df97057-kube-api-access-46n2s\") pod \"llmisvc-controller-manager-777c6bd7d5-nfbc7\" (UID: \"d7818d3c-4200-40bd-a528-f4266df97057\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:44:22.467051 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:22.467007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7818d3c-4200-40bd-a528-f4266df97057-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nfbc7\" (UID: \"d7818d3c-4200-40bd-a528-f4266df97057\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:44:22.469761 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:22.469730    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7818d3c-4200-40bd-a528-f4266df97057-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nfbc7\" (UID: \"d7818d3c-4200-40bd-a528-f4266df97057\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:44:22.617365 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:22.617318    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:44:22.901953 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:22.901925    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"]
Sep 11 17:44:22.903913 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:44:22.903879    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd7818d3c_4200_40bd_a528_f4266df97057.slice/crio-b6458e5454decdb52a47f998ca55c21b5da15a0f7dfe036581848de0ae4a01c0 WatchSource:0}: Error finding container b6458e5454decdb52a47f998ca55c21b5da15a0f7dfe036581848de0ae4a01c0: Status 404 returned error can't find the container with id b6458e5454decdb52a47f998ca55c21b5da15a0f7dfe036581848de0ae4a01c0
Sep 11 17:44:23.060258 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:23.060220    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7" event={"ID":"d7818d3c-4200-40bd-a528-f4266df97057","Type":"ContainerStarted","Data":"b6458e5454decdb52a47f998ca55c21b5da15a0f7dfe036581848de0ae4a01c0"}
Sep 11 17:44:27.078682 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:27.078629    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7" event={"ID":"d7818d3c-4200-40bd-a528-f4266df97057","Type":"ContainerStarted","Data":"b8f20166379681177bf4cf042debfb6793cba489d869507cb4ef19335d14ec2f"}
Sep 11 17:44:27.079146 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:27.078767    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:44:27.101292 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:27.101239    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7" podStartSLOduration=2.999194136 podStartE2EDuration="6.101221138s" totalImagesPullingTime="3.102027002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:44:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:44:27.099496388 +0000 UTC m=+334.987674734" watchObservedRunningTime="2026-09-11 17:44:27.101221138 +0000 UTC m=+334.989399483"
Sep 11 17:44:58.084742 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:44:58.084712    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nfbc7"
Sep 11 17:45:01.684952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.684911    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58fffc6dc9-ml9f9"]
Sep 11 17:45:01.692884 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.692864    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.695949 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.695917    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 17:45:01.696129 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.695964    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 17:45:01.697333 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.697265    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 17:45:01.697454 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.697356    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 17:45:01.697685 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.697661    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 17:45:01.697933 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.697915    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 17:45:01.697999 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.697933    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 17:45:01.697999 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.697917    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-569cd\""
Sep 11 17:45:01.701057 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.701034    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58fffc6dc9-ml9f9"]
Sep 11 17:45:01.702679 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.702658    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 17:45:01.805425 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.805385    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-console-oauth-config\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.805425 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.805431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-oauth-serving-cert\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.805668 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.805468    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-service-ca\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.805668 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.805511    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-console-serving-cert\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.805668 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.805547    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-console-config\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.805668 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.805589    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-trusted-ca-bundle\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.805668 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.805655    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbpxq\" (UniqueName: \"kubernetes.io/projected/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-kube-api-access-tbpxq\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.906771 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.906727    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-console-oauth-config\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.906952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.906781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-oauth-serving-cert\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.906952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.906825    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-service-ca\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.906952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.906849    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-console-serving-cert\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.907203 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.906961    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-console-config\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.907203 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.906993    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-trusted-ca-bundle\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.907203 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.907023    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbpxq\" (UniqueName: \"kubernetes.io/projected/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-kube-api-access-tbpxq\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.907674 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.907651    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-service-ca\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.907938 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.907921    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-trusted-ca-bundle\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.907979 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.907941    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-oauth-serving-cert\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.907979 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.907960    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-console-config\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.909494 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.909469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-console-oauth-config\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.909583 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.909516    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-console-serving-cert\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:01.917595 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:01.917568    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbpxq\" (UniqueName: \"kubernetes.io/projected/73abb0f1-f5a3-4a5c-8e73-a364a985f22c-kube-api-access-tbpxq\") pod \"console-58fffc6dc9-ml9f9\" (UID: \"73abb0f1-f5a3-4a5c-8e73-a364a985f22c\") " pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:02.003213 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:02.003104    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:02.279052 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:02.279023    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58fffc6dc9-ml9f9"]
Sep 11 17:45:02.281768 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:45:02.281737    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73abb0f1_f5a3_4a5c_8e73_a364a985f22c.slice/crio-76b9c44fae6fe2f220035a19f8de936f7082027d148a4132826aff1278ef0c0a WatchSource:0}: Error finding container 76b9c44fae6fe2f220035a19f8de936f7082027d148a4132826aff1278ef0c0a: Status 404 returned error can't find the container with id 76b9c44fae6fe2f220035a19f8de936f7082027d148a4132826aff1278ef0c0a
Sep 11 17:45:03.225875 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:03.225829    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58fffc6dc9-ml9f9" event={"ID":"73abb0f1-f5a3-4a5c-8e73-a364a985f22c","Type":"ContainerStarted","Data":"5707e02184aed6665b5eac6db74c71e08138a520d03e521e009f0b291e14bdf9"}
Sep 11 17:45:03.226390 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:03.225909    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58fffc6dc9-ml9f9" event={"ID":"73abb0f1-f5a3-4a5c-8e73-a364a985f22c","Type":"ContainerStarted","Data":"76b9c44fae6fe2f220035a19f8de936f7082027d148a4132826aff1278ef0c0a"}
Sep 11 17:45:03.247748 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:03.247691    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58fffc6dc9-ml9f9" podStartSLOduration=2.2476749050000002 podStartE2EDuration="2.247674905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:45:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:45:03.245903378 +0000 UTC m=+371.134081725" watchObservedRunningTime="2026-09-11 17:45:03.247674905 +0000 UTC m=+371.135853304"
Sep 11 17:45:12.003758 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:12.003707    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:12.004355 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:12.003781    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:12.008836 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:12.008808    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:12.268068 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:12.267978    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58fffc6dc9-ml9f9"
Sep 11 17:45:58.376991 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.376955    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl"]
Sep 11 17:45:58.380328 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.380311    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl"
Sep 11 17:45:58.384524 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.384504    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-b5q2g\""
Sep 11 17:45:58.384625 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.384504    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 17:45:58.385822 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.385803    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 17:45:58.396810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.396786    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl"]
Sep 11 17:45:58.489812 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.489768    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mbmd\" (UniqueName: \"kubernetes.io/projected/bf2615bd-3591-4f82-91c6-728152eab5c0-kube-api-access-9mbmd\") pod \"limitador-operator-controller-manager-74495c69f8-4txbl\" (UID: \"bf2615bd-3591-4f82-91c6-728152eab5c0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl"
Sep 11 17:45:58.590217 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.590190    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mbmd\" (UniqueName: \"kubernetes.io/projected/bf2615bd-3591-4f82-91c6-728152eab5c0-kube-api-access-9mbmd\") pod \"limitador-operator-controller-manager-74495c69f8-4txbl\" (UID: \"bf2615bd-3591-4f82-91c6-728152eab5c0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl"
Sep 11 17:45:58.603213 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.603194    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mbmd\" (UniqueName: \"kubernetes.io/projected/bf2615bd-3591-4f82-91c6-728152eab5c0-kube-api-access-9mbmd\") pod \"limitador-operator-controller-manager-74495c69f8-4txbl\" (UID: \"bf2615bd-3591-4f82-91c6-728152eab5c0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl"
Sep 11 17:45:58.690424 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.690390    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl"
Sep 11 17:45:58.966226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:58.966201    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl"]
Sep 11 17:45:58.968247 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:45:58.968222    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf2615bd_3591_4f82_91c6_728152eab5c0.slice/crio-4bca9b47b3879a811a237de609860944c3df59c8f5cca289a8ea0faf28e8e90e WatchSource:0}: Error finding container 4bca9b47b3879a811a237de609860944c3df59c8f5cca289a8ea0faf28e8e90e: Status 404 returned error can't find the container with id 4bca9b47b3879a811a237de609860944c3df59c8f5cca289a8ea0faf28e8e90e
Sep 11 17:45:59.462115 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:45:59.462060    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl" event={"ID":"bf2615bd-3591-4f82-91c6-728152eab5c0","Type":"ContainerStarted","Data":"4bca9b47b3879a811a237de609860944c3df59c8f5cca289a8ea0faf28e8e90e"}
Sep 11 17:46:01.472175 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:01.472138    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl" event={"ID":"bf2615bd-3591-4f82-91c6-728152eab5c0","Type":"ContainerStarted","Data":"150e1649586f37343a529fef60043321c564410d4bb56b305cc410bb1e2930c5"}
Sep 11 17:46:01.472579 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:01.472287    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl"
Sep 11 17:46:01.493323 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:01.493278    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl" podStartSLOduration=1.361967333 podStartE2EDuration="3.493263821s" totalImagesPullingTime="2.131296488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:45:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:46:01.492387871 +0000 UTC m=+429.380566217" watchObservedRunningTime="2026-09-11 17:46:01.493263821 +0000 UTC m=+429.381442166"
Sep 11 17:46:12.479577 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:12.479541    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4txbl"
Sep 11 17:46:41.393262 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.393217    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-d6r68"]
Sep 11 17:46:41.398937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.398911    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68"
Sep 11 17:46:41.401921 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.401869    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sprwn\""
Sep 11 17:46:41.403730 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.403694    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-d6r68"]
Sep 11 17:46:41.460654 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.460614    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sczm\" (UniqueName: \"kubernetes.io/projected/2dad42e7-baf1-4c13-a92e-92f5933ed6a0-kube-api-access-2sczm\") pod \"authorino-7bcfd8f9cd-d6r68\" (UID: \"2dad42e7-baf1-4c13-a92e-92f5933ed6a0\") " pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68"
Sep 11 17:46:41.517860 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.517824    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt648"]
Sep 11 17:46:41.520622 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.520606    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kt648"
Sep 11 17:46:41.528718 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.528691    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt648"]
Sep 11 17:46:41.561178 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.561140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sczm\" (UniqueName: \"kubernetes.io/projected/2dad42e7-baf1-4c13-a92e-92f5933ed6a0-kube-api-access-2sczm\") pod \"authorino-7bcfd8f9cd-d6r68\" (UID: \"2dad42e7-baf1-4c13-a92e-92f5933ed6a0\") " pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68"
Sep 11 17:46:41.569901 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.569868    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sczm\" (UniqueName: \"kubernetes.io/projected/2dad42e7-baf1-4c13-a92e-92f5933ed6a0-kube-api-access-2sczm\") pod \"authorino-7bcfd8f9cd-d6r68\" (UID: \"2dad42e7-baf1-4c13-a92e-92f5933ed6a0\") " pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68"
Sep 11 17:46:41.662274 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.662185    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9d5r\" (UniqueName: \"kubernetes.io/projected/ddbf6983-4c09-4477-8be1-feee4b756da6-kube-api-access-n9d5r\") pod \"authorino-59c974bd48-kt648\" (UID: \"ddbf6983-4c09-4477-8be1-feee4b756da6\") " pod="kuadrant-system/authorino-59c974bd48-kt648"
Sep 11 17:46:41.709151 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.709109    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68"
Sep 11 17:46:41.763622 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.763594    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9d5r\" (UniqueName: \"kubernetes.io/projected/ddbf6983-4c09-4477-8be1-feee4b756da6-kube-api-access-n9d5r\") pod \"authorino-59c974bd48-kt648\" (UID: \"ddbf6983-4c09-4477-8be1-feee4b756da6\") " pod="kuadrant-system/authorino-59c974bd48-kt648"
Sep 11 17:46:41.775030 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.775002    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9d5r\" (UniqueName: \"kubernetes.io/projected/ddbf6983-4c09-4477-8be1-feee4b756da6-kube-api-access-n9d5r\") pod \"authorino-59c974bd48-kt648\" (UID: \"ddbf6983-4c09-4477-8be1-feee4b756da6\") " pod="kuadrant-system/authorino-59c974bd48-kt648"
Sep 11 17:46:41.829948 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.829910    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kt648"
Sep 11 17:46:41.978365 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:41.978335    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-d6r68"]
Sep 11 17:46:41.980397 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:46:41.980368    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dad42e7_baf1_4c13_a92e_92f5933ed6a0.slice/crio-678d1a8ae62e25a590cc636ed7030ba473051b348cd78f2e755b75a65a2598da WatchSource:0}: Error finding container 678d1a8ae62e25a590cc636ed7030ba473051b348cd78f2e755b75a65a2598da: Status 404 returned error can't find the container with id 678d1a8ae62e25a590cc636ed7030ba473051b348cd78f2e755b75a65a2598da
Sep 11 17:46:42.097405 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:42.097356    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt648"]
Sep 11 17:46:42.099978 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:46:42.099938    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddbf6983_4c09_4477_8be1_feee4b756da6.slice/crio-3850efba21b04811c2a4b65b69b9cfb8e3256b8169bbdaac955d1c0233bb25e9 WatchSource:0}: Error finding container 3850efba21b04811c2a4b65b69b9cfb8e3256b8169bbdaac955d1c0233bb25e9: Status 404 returned error can't find the container with id 3850efba21b04811c2a4b65b69b9cfb8e3256b8169bbdaac955d1c0233bb25e9
Sep 11 17:46:42.642023 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:42.641987    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kt648" event={"ID":"ddbf6983-4c09-4477-8be1-feee4b756da6","Type":"ContainerStarted","Data":"3850efba21b04811c2a4b65b69b9cfb8e3256b8169bbdaac955d1c0233bb25e9"}
Sep 11 17:46:42.643130 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:42.643107    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68" event={"ID":"2dad42e7-baf1-4c13-a92e-92f5933ed6a0","Type":"ContainerStarted","Data":"678d1a8ae62e25a590cc636ed7030ba473051b348cd78f2e755b75a65a2598da"}
Sep 11 17:46:44.654126 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:44.654075    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68" event={"ID":"2dad42e7-baf1-4c13-a92e-92f5933ed6a0","Type":"ContainerStarted","Data":"109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16"}
Sep 11 17:46:44.655544 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:44.655522    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kt648" event={"ID":"ddbf6983-4c09-4477-8be1-feee4b756da6","Type":"ContainerStarted","Data":"726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d"}
Sep 11 17:46:44.680524 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:44.680474    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68" podStartSLOduration=1.874658677 podStartE2EDuration="3.680457306s" totalImagesPullingTime="1.805798629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:46:44.67940905 +0000 UTC m=+472.567587396" watchObservedRunningTime="2026-09-11 17:46:44.680457306 +0000 UTC m=+472.568635652"
Sep 11 17:46:44.705706 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:44.705652    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-kt648" podStartSLOduration=2.015675816 podStartE2EDuration="3.705632873s" totalImagesPullingTime="1.689957057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:46:44.704338391 +0000 UTC m=+472.592516740" watchObservedRunningTime="2026-09-11 17:46:44.705632873 +0000 UTC m=+472.593811219"
Sep 11 17:46:44.747064 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:44.747020    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-d6r68"]
Sep 11 17:46:46.664326 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:46.664280    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68" podUID="2dad42e7-baf1-4c13-a92e-92f5933ed6a0" containerName="authorino" containerID="cri-o://109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16" gracePeriod=30
Sep 11 17:46:46.897286 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:46.897260    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68"
Sep 11 17:46:47.009676 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.009590    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sczm\" (UniqueName: \"kubernetes.io/projected/2dad42e7-baf1-4c13-a92e-92f5933ed6a0-kube-api-access-2sczm\") pod \"2dad42e7-baf1-4c13-a92e-92f5933ed6a0\" (UID: \"2dad42e7-baf1-4c13-a92e-92f5933ed6a0\") "
Sep 11 17:46:47.012049 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.012014    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dad42e7-baf1-4c13-a92e-92f5933ed6a0-kube-api-access-2sczm" (OuterVolumeSpecName: "kube-api-access-2sczm") pod "2dad42e7-baf1-4c13-a92e-92f5933ed6a0" (UID: "2dad42e7-baf1-4c13-a92e-92f5933ed6a0"). InnerVolumeSpecName "kube-api-access-2sczm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:46:47.110830 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.110789    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sczm\" (UniqueName: \"kubernetes.io/projected/2dad42e7-baf1-4c13-a92e-92f5933ed6a0-kube-api-access-2sczm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:46:47.669754 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.669714    2582 generic.go:356] "Generic (PLEG): container finished" podID="2dad42e7-baf1-4c13-a92e-92f5933ed6a0" containerID="109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16" exitCode=0
Sep 11 17:46:47.670246 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.669761    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68"
Sep 11 17:46:47.670246 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.669805    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68" event={"ID":"2dad42e7-baf1-4c13-a92e-92f5933ed6a0","Type":"ContainerDied","Data":"109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16"}
Sep 11 17:46:47.670246 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.669856    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-d6r68" event={"ID":"2dad42e7-baf1-4c13-a92e-92f5933ed6a0","Type":"ContainerDied","Data":"678d1a8ae62e25a590cc636ed7030ba473051b348cd78f2e755b75a65a2598da"}
Sep 11 17:46:47.670246 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.669880    2582 scope.go:117] "RemoveContainer" containerID="109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16"
Sep 11 17:46:47.678472 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.678455    2582 scope.go:117] "RemoveContainer" containerID="109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16"
Sep 11 17:46:47.678746 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:46:47.678727    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16\": container with ID starting with 109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16 not found: ID does not exist" containerID="109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16"
Sep 11 17:46:47.678810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.678756    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16"} err="failed to get container status \"109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16\": rpc error: code = NotFound desc = could not find container \"109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16\": container with ID starting with 109326799bcd1eba9fdf41119ec61ace2595119cf71cf5cb7fa6c50961c45c16 not found: ID does not exist"
Sep 11 17:46:47.692639 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.692613    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-d6r68"]
Sep 11 17:46:47.698521 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:47.698500    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-d6r68"]
Sep 11 17:46:48.633473 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:46:48.633440    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dad42e7-baf1-4c13-a92e-92f5933ed6a0" path="/var/lib/kubelet/pods/2dad42e7-baf1-4c13-a92e-92f5933ed6a0/volumes"
Sep 11 17:47:07.822025 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:07.821987    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-tqjtz"]
Sep 11 17:47:07.822491 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:07.822325    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dad42e7-baf1-4c13-a92e-92f5933ed6a0" containerName="authorino"
Sep 11 17:47:07.822491 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:07.822337    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dad42e7-baf1-4c13-a92e-92f5933ed6a0" containerName="authorino"
Sep 11 17:47:07.822491 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:07.822386    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dad42e7-baf1-4c13-a92e-92f5933ed6a0" containerName="authorino"
Sep 11 17:47:07.824930 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:07.824914    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-tqjtz"
Sep 11 17:47:07.842828 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:07.842798    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-tqjtz"]
Sep 11 17:47:07.904057 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:07.904020    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtx29\" (UniqueName: \"kubernetes.io/projected/b6607b0e-d3ab-4b4b-a62b-771fc0c21185-kube-api-access-gtx29\") pod \"authorino-557f876f69-tqjtz\" (UID: \"b6607b0e-d3ab-4b4b-a62b-771fc0c21185\") " pod="kuadrant-system/authorino-557f876f69-tqjtz"
Sep 11 17:47:08.004184 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.004147    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-tqjtz"]
Sep 11 17:47:08.004410 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:47:08.004391    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gtx29], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-tqjtz" podUID="b6607b0e-d3ab-4b4b-a62b-771fc0c21185"
Sep 11 17:47:08.004468 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.004449    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtx29\" (UniqueName: \"kubernetes.io/projected/b6607b0e-d3ab-4b4b-a62b-771fc0c21185-kube-api-access-gtx29\") pod \"authorino-557f876f69-tqjtz\" (UID: \"b6607b0e-d3ab-4b4b-a62b-771fc0c21185\") " pod="kuadrant-system/authorino-557f876f69-tqjtz"
Sep 11 17:47:08.032757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.032723    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtx29\" (UniqueName: \"kubernetes.io/projected/b6607b0e-d3ab-4b4b-a62b-771fc0c21185-kube-api-access-gtx29\") pod \"authorino-557f876f69-tqjtz\" (UID: \"b6607b0e-d3ab-4b4b-a62b-771fc0c21185\") " pod="kuadrant-system/authorino-557f876f69-tqjtz"
Sep 11 17:47:08.059786 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.059749    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-2vz9v"]
Sep 11 17:47:08.062200 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.062182    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-2vz9v"
Sep 11 17:47:08.067046 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.067019    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 17:47:08.078969 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.078907    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-2vz9v"]
Sep 11 17:47:08.105455 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.105420    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/82e02660-36da-4139-a1e7-ddc70607bbb0-tls-cert\") pod \"authorino-6d4dfc649f-2vz9v\" (UID: \"82e02660-36da-4139-a1e7-ddc70607bbb0\") " pod="kuadrant-system/authorino-6d4dfc649f-2vz9v"
Sep 11 17:47:08.105645 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.105468    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hmnz\" (UniqueName: \"kubernetes.io/projected/82e02660-36da-4139-a1e7-ddc70607bbb0-kube-api-access-7hmnz\") pod \"authorino-6d4dfc649f-2vz9v\" (UID: \"82e02660-36da-4139-a1e7-ddc70607bbb0\") " pod="kuadrant-system/authorino-6d4dfc649f-2vz9v"
Sep 11 17:47:08.123450 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.123417    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-2vz9v"]
Sep 11 17:47:08.123679 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:47:08.123660    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7hmnz tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-2vz9v" podUID="82e02660-36da-4139-a1e7-ddc70607bbb0"
Sep 11 17:47:08.159625 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.159585    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66cd586c6-z2wcb"]
Sep 11 17:47:08.161813 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.161793    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66cd586c6-z2wcb"
Sep 11 17:47:08.173862 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.173836    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66cd586c6-z2wcb"]
Sep 11 17:47:08.205859 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.205819    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/82e02660-36da-4139-a1e7-ddc70607bbb0-tls-cert\") pod \"authorino-6d4dfc649f-2vz9v\" (UID: \"82e02660-36da-4139-a1e7-ddc70607bbb0\") " pod="kuadrant-system/authorino-6d4dfc649f-2vz9v"
Sep 11 17:47:08.205859 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.205859    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hmnz\" (UniqueName: \"kubernetes.io/projected/82e02660-36da-4139-a1e7-ddc70607bbb0-kube-api-access-7hmnz\") pod \"authorino-6d4dfc649f-2vz9v\" (UID: \"82e02660-36da-4139-a1e7-ddc70607bbb0\") " pod="kuadrant-system/authorino-6d4dfc649f-2vz9v"
Sep 11 17:47:08.206074 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.205893    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b423686-c0b4-46ed-b316-e63ec050a57c-tls-cert\") pod \"authorino-66cd586c6-z2wcb\" (UID: \"4b423686-c0b4-46ed-b316-e63ec050a57c\") " pod="kuadrant-system/authorino-66cd586c6-z2wcb"
Sep 11 17:47:08.206074 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.205931    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dktd\" (UniqueName: \"kubernetes.io/projected/4b423686-c0b4-46ed-b316-e63ec050a57c-kube-api-access-5dktd\") pod \"authorino-66cd586c6-z2wcb\" (UID: \"4b423686-c0b4-46ed-b316-e63ec050a57c\") " pod="kuadrant-system/authorino-66cd586c6-z2wcb"
Sep 11 17:47:08.208358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.208338    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/82e02660-36da-4139-a1e7-ddc70607bbb0-tls-cert\") pod \"authorino-6d4dfc649f-2vz9v\" (UID: \"82e02660-36da-4139-a1e7-ddc70607bbb0\") " pod="kuadrant-system/authorino-6d4dfc649f-2vz9v"
Sep 11 17:47:08.215364 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.215340    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hmnz\" (UniqueName: \"kubernetes.io/projected/82e02660-36da-4139-a1e7-ddc70607bbb0-kube-api-access-7hmnz\") pod \"authorino-6d4dfc649f-2vz9v\" (UID: \"82e02660-36da-4139-a1e7-ddc70607bbb0\") " pod="kuadrant-system/authorino-6d4dfc649f-2vz9v"
Sep 11 17:47:08.306455 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.306417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b423686-c0b4-46ed-b316-e63ec050a57c-tls-cert\") pod \"authorino-66cd586c6-z2wcb\" (UID: \"4b423686-c0b4-46ed-b316-e63ec050a57c\") " pod="kuadrant-system/authorino-66cd586c6-z2wcb"
Sep 11 17:47:08.306621 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.306471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dktd\" (UniqueName: \"kubernetes.io/projected/4b423686-c0b4-46ed-b316-e63ec050a57c-kube-api-access-5dktd\") pod \"authorino-66cd586c6-z2wcb\" (UID: \"4b423686-c0b4-46ed-b316-e63ec050a57c\") " pod="kuadrant-system/authorino-66cd586c6-z2wcb"
Sep 11 17:47:08.308951 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.308931    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b423686-c0b4-46ed-b316-e63ec050a57c-tls-cert\") pod \"authorino-66cd586c6-z2wcb\" (UID: \"4b423686-c0b4-46ed-b316-e63ec050a57c\") " pod="kuadrant-system/authorino-66cd586c6-z2wcb"
Sep 11 17:47:08.317187 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.317158    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dktd\" (UniqueName: \"kubernetes.io/projected/4b423686-c0b4-46ed-b316-e63ec050a57c-kube-api-access-5dktd\") pod \"authorino-66cd586c6-z2wcb\" (UID: \"4b423686-c0b4-46ed-b316-e63ec050a57c\") " pod="kuadrant-system/authorino-66cd586c6-z2wcb"
Sep 11 17:47:08.470583 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.470542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66cd586c6-z2wcb"
Sep 11 17:47:08.741103 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.741011    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66cd586c6-z2wcb"]
Sep 11 17:47:08.743977 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:47:08.743953    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b423686_c0b4_46ed_b316_e63ec050a57c.slice/crio-f4ea6c556e346d1d5f923e1e658c8bb263e891f738556b73e70d6018ccfd1cde WatchSource:0}: Error finding container f4ea6c556e346d1d5f923e1e658c8bb263e891f738556b73e70d6018ccfd1cde: Status 404 returned error can't find the container with id f4ea6c556e346d1d5f923e1e658c8bb263e891f738556b73e70d6018ccfd1cde
Sep 11 17:47:08.765664 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.765639    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-2vz9v"
Sep 11 17:47:08.765838 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.765676    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66cd586c6-z2wcb" event={"ID":"4b423686-c0b4-46ed-b316-e63ec050a57c","Type":"ContainerStarted","Data":"f4ea6c556e346d1d5f923e1e658c8bb263e891f738556b73e70d6018ccfd1cde"}
Sep 11 17:47:08.765838 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.765713    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-tqjtz"
Sep 11 17:47:08.772715 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.772690    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-2vz9v"
Sep 11 17:47:08.774934 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.774916    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-tqjtz"
Sep 11 17:47:08.810522 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.810492    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtx29\" (UniqueName: \"kubernetes.io/projected/b6607b0e-d3ab-4b4b-a62b-771fc0c21185-kube-api-access-gtx29\") pod \"b6607b0e-d3ab-4b4b-a62b-771fc0c21185\" (UID: \"b6607b0e-d3ab-4b4b-a62b-771fc0c21185\") "
Sep 11 17:47:08.810694 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.810582    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/82e02660-36da-4139-a1e7-ddc70607bbb0-tls-cert\") pod \"82e02660-36da-4139-a1e7-ddc70607bbb0\" (UID: \"82e02660-36da-4139-a1e7-ddc70607bbb0\") "
Sep 11 17:47:08.810694 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.810652    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hmnz\" (UniqueName: \"kubernetes.io/projected/82e02660-36da-4139-a1e7-ddc70607bbb0-kube-api-access-7hmnz\") pod \"82e02660-36da-4139-a1e7-ddc70607bbb0\" (UID: \"82e02660-36da-4139-a1e7-ddc70607bbb0\") "
Sep 11 17:47:08.812805 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.812771    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e02660-36da-4139-a1e7-ddc70607bbb0-kube-api-access-7hmnz" (OuterVolumeSpecName: "kube-api-access-7hmnz") pod "82e02660-36da-4139-a1e7-ddc70607bbb0" (UID: "82e02660-36da-4139-a1e7-ddc70607bbb0"). InnerVolumeSpecName "kube-api-access-7hmnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:47:08.812908 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.812815    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6607b0e-d3ab-4b4b-a62b-771fc0c21185-kube-api-access-gtx29" (OuterVolumeSpecName: "kube-api-access-gtx29") pod "b6607b0e-d3ab-4b4b-a62b-771fc0c21185" (UID: "b6607b0e-d3ab-4b4b-a62b-771fc0c21185"). InnerVolumeSpecName "kube-api-access-gtx29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:47:08.812908 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.812819    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82e02660-36da-4139-a1e7-ddc70607bbb0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "82e02660-36da-4139-a1e7-ddc70607bbb0" (UID: "82e02660-36da-4139-a1e7-ddc70607bbb0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:47:08.911614 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.911560    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtx29\" (UniqueName: \"kubernetes.io/projected/b6607b0e-d3ab-4b4b-a62b-771fc0c21185-kube-api-access-gtx29\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:47:08.911614 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.911606    2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/82e02660-36da-4139-a1e7-ddc70607bbb0-tls-cert\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:47:08.911614 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:08.911616    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hmnz\" (UniqueName: \"kubernetes.io/projected/82e02660-36da-4139-a1e7-ddc70607bbb0-kube-api-access-7hmnz\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:47:09.772357 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:09.772314    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66cd586c6-z2wcb" event={"ID":"4b423686-c0b4-46ed-b316-e63ec050a57c","Type":"ContainerStarted","Data":"c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa"}
Sep 11 17:47:09.772357 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:09.772353    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-2vz9v"
Sep 11 17:47:09.772601 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:09.772337    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-tqjtz"
Sep 11 17:47:09.795510 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:09.795454    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66cd586c6-z2wcb" podStartSLOduration=1.247418494 podStartE2EDuration="1.795440224s" totalImagesPullingTime="548.02173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:47:09.793151842 +0000 UTC m=+497.681330187" watchObservedRunningTime="2026-09-11 17:47:09.795440224 +0000 UTC m=+497.683618570"
Sep 11 17:47:09.824408 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:09.824373    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt648"]
Sep 11 17:47:09.824600 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:09.824577    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-kt648" podUID="ddbf6983-4c09-4477-8be1-feee4b756da6" containerName="authorino" containerID="cri-o://726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d" gracePeriod=30
Sep 11 17:47:09.829726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:09.829704    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-2vz9v"]
Sep 11 17:47:09.843165 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:09.843128    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-2vz9v"]
Sep 11 17:47:09.871638 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:09.871602    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-tqjtz"]
Sep 11 17:47:09.877568 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:09.877539    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-tqjtz"]
Sep 11 17:47:10.057327 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.057302    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kt648"
Sep 11 17:47:10.122891 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.122862    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9d5r\" (UniqueName: \"kubernetes.io/projected/ddbf6983-4c09-4477-8be1-feee4b756da6-kube-api-access-n9d5r\") pod \"ddbf6983-4c09-4477-8be1-feee4b756da6\" (UID: \"ddbf6983-4c09-4477-8be1-feee4b756da6\") "
Sep 11 17:47:10.125112 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.125075    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddbf6983-4c09-4477-8be1-feee4b756da6-kube-api-access-n9d5r" (OuterVolumeSpecName: "kube-api-access-n9d5r") pod "ddbf6983-4c09-4477-8be1-feee4b756da6" (UID: "ddbf6983-4c09-4477-8be1-feee4b756da6"). InnerVolumeSpecName "kube-api-access-n9d5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:47:10.223612 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.223577    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9d5r\" (UniqueName: \"kubernetes.io/projected/ddbf6983-4c09-4477-8be1-feee4b756da6-kube-api-access-n9d5r\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:47:10.633061 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.633026    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82e02660-36da-4139-a1e7-ddc70607bbb0" path="/var/lib/kubelet/pods/82e02660-36da-4139-a1e7-ddc70607bbb0/volumes"
Sep 11 17:47:10.633358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.633340    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6607b0e-d3ab-4b4b-a62b-771fc0c21185" path="/var/lib/kubelet/pods/b6607b0e-d3ab-4b4b-a62b-771fc0c21185/volumes"
Sep 11 17:47:10.778363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.778326    2582 generic.go:356] "Generic (PLEG): container finished" podID="ddbf6983-4c09-4477-8be1-feee4b756da6" containerID="726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d" exitCode=0
Sep 11 17:47:10.778363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.778366    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kt648"
Sep 11 17:47:10.778566 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.778408    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kt648" event={"ID":"ddbf6983-4c09-4477-8be1-feee4b756da6","Type":"ContainerDied","Data":"726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d"}
Sep 11 17:47:10.778566 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.778451    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kt648" event={"ID":"ddbf6983-4c09-4477-8be1-feee4b756da6","Type":"ContainerDied","Data":"3850efba21b04811c2a4b65b69b9cfb8e3256b8169bbdaac955d1c0233bb25e9"}
Sep 11 17:47:10.778566 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.778469    2582 scope.go:117] "RemoveContainer" containerID="726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d"
Sep 11 17:47:10.786515 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.786497    2582 scope.go:117] "RemoveContainer" containerID="726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d"
Sep 11 17:47:10.786809 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:47:10.786791    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d\": container with ID starting with 726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d not found: ID does not exist" containerID="726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d"
Sep 11 17:47:10.786857 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.786821    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d"} err="failed to get container status \"726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d\": rpc error: code = NotFound desc = could not find container \"726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d\": container with ID starting with 726c39af1fbd38424795cd5e2b79c26ad35f63065af836c5f536593b8f78ce7d not found: ID does not exist"
Sep 11 17:47:10.799450 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.799421    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt648"]
Sep 11 17:47:10.802009 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:10.801986    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kt648"]
Sep 11 17:47:12.632998 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:47:12.632960    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddbf6983-4c09-4477-8be1-feee4b756da6" path="/var/lib/kubelet/pods/ddbf6983-4c09-4477-8be1-feee4b756da6/volumes"
Sep 11 17:48:16.387623 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.387589    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"]
Sep 11 17:48:16.388173 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.388062    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddbf6983-4c09-4477-8be1-feee4b756da6" containerName="authorino"
Sep 11 17:48:16.388173 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.388094    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddbf6983-4c09-4477-8be1-feee4b756da6" containerName="authorino"
Sep 11 17:48:16.388173 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.388152    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddbf6983-4c09-4477-8be1-feee4b756da6" containerName="authorino"
Sep 11 17:48:16.391069 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.391048    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.394023 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.393991    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xdfcz\""
Sep 11 17:48:16.394176 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.394068    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 17:48:16.395429 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.395400    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 17:48:16.395534 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.395447    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 17:48:16.409136 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.409106    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"]
Sep 11 17:48:16.488744 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.488702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzc7w\" (UniqueName: \"kubernetes.io/projected/3b5f86cc-32dc-453d-97f0-a16370fe385f-kube-api-access-jzc7w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.488934 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.488769    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.488934 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.488796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.488934 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.488836    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.488934 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.488887    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.488934 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.488916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5f86cc-32dc-453d-97f0-a16370fe385f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.590031 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.589985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.590031 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.590040    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5f86cc-32dc-453d-97f0-a16370fe385f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.590316 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.590072    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzc7w\" (UniqueName: \"kubernetes.io/projected/3b5f86cc-32dc-453d-97f0-a16370fe385f-kube-api-access-jzc7w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.590316 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.590147    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.590316 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.590175    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.590316 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.590200    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.590631 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.590606    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.590705 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.590627    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.590705 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.590677    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.592606 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.592584    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5f86cc-32dc-453d-97f0-a16370fe385f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.592791 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.592775    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5f86cc-32dc-453d-97f0-a16370fe385f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.598909 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.598888    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzc7w\" (UniqueName: \"kubernetes.io/projected/3b5f86cc-32dc-453d-97f0-a16370fe385f-kube-api-access-jzc7w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p99l8\" (UID: \"3b5f86cc-32dc-453d-97f0-a16370fe385f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.701835 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.701801    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:16.977422 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:16.977338    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"]
Sep 11 17:48:16.980220 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:48:16.980188    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5f86cc_32dc_453d_97f0_a16370fe385f.slice/crio-cfc3bdc9d2cc5ffae74a4d384de2e5cfa3c3f26b6710a5f638dfad018ac77a92 WatchSource:0}: Error finding container cfc3bdc9d2cc5ffae74a4d384de2e5cfa3c3f26b6710a5f638dfad018ac77a92: Status 404 returned error can't find the container with id cfc3bdc9d2cc5ffae74a4d384de2e5cfa3c3f26b6710a5f638dfad018ac77a92
Sep 11 17:48:17.064352 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:17.064313    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8" event={"ID":"3b5f86cc-32dc-453d-97f0-a16370fe385f","Type":"ContainerStarted","Data":"cfc3bdc9d2cc5ffae74a4d384de2e5cfa3c3f26b6710a5f638dfad018ac77a92"}
Sep 11 17:48:19.074122 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:19.074072    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8" event={"ID":"3b5f86cc-32dc-453d-97f0-a16370fe385f","Type":"ContainerStarted","Data":"e6001ee87919d9a87148799e86be024d960ecc0b619b87b7a6015aa10c6f8350"}
Sep 11 17:48:19.107327 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:19.107268    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8" podStartSLOduration=1.982681082 podStartE2EDuration="3.107252275s" totalImagesPullingTime="1.124571193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:48:19.106463284 +0000 UTC m=+566.994641631" watchObservedRunningTime="2026-09-11 17:48:19.107252275 +0000 UTC m=+566.995430621"
Sep 11 17:48:26.702699 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:26.702659    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:26.703246 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:26.702729    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:26.715774 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:26.715746    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:27.122267 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:27.122174    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p99l8"
Sep 11 17:48:44.274640 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.274606    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"]
Sep 11 17:48:44.277891 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.277866    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.280581 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.280557    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 11 17:48:44.289837 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.289808    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"]
Sep 11 17:48:44.439532 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.439483    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.439532 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.439534    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51cdc09f-d524-47d2-883b-f667124d937d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.439768 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.439648    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.439768 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.439677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f64sk\" (UniqueName: \"kubernetes.io/projected/51cdc09f-d524-47d2-883b-f667124d937d-kube-api-access-f64sk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.439768 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.439714    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.439881 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.439770    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.540843 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.540757    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.540843 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.540812    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.541052 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.540846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.541052 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.540872    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51cdc09f-d524-47d2-883b-f667124d937d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.541052 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.540908    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.541052 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.540934    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f64sk\" (UniqueName: \"kubernetes.io/projected/51cdc09f-d524-47d2-883b-f667124d937d-kube-api-access-f64sk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.541360 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.541321    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.541360 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.541363    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.541360 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.541330    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.543348 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.543328    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51cdc09f-d524-47d2-883b-f667124d937d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.543533 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.543514    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51cdc09f-d524-47d2-883b-f667124d937d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.549389 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.549363    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f64sk\" (UniqueName: \"kubernetes.io/projected/51cdc09f-d524-47d2-883b-f667124d937d-kube-api-access-f64sk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-m9fwc\" (UID: \"51cdc09f-d524-47d2-883b-f667124d937d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.589297 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.589248    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:44.865100 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:44.865055    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"]
Sep 11 17:48:44.867676 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:48:44.867642    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51cdc09f_d524_47d2_883b_f667124d937d.slice/crio-8eaee10add4cd40259c97ad6f9b70d6c18500d3ad4a5f4a92683895f72497df7 WatchSource:0}: Error finding container 8eaee10add4cd40259c97ad6f9b70d6c18500d3ad4a5f4a92683895f72497df7: Status 404 returned error can't find the container with id 8eaee10add4cd40259c97ad6f9b70d6c18500d3ad4a5f4a92683895f72497df7
Sep 11 17:48:45.188794 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:45.188756    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc" event={"ID":"51cdc09f-d524-47d2-883b-f667124d937d","Type":"ContainerStarted","Data":"16b979132ed6cc4c689683e498536442e95c0287eed3187f8851bdde27cfd1c8"}
Sep 11 17:48:45.188962 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:45.188838    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc" event={"ID":"51cdc09f-d524-47d2-883b-f667124d937d","Type":"ContainerStarted","Data":"8eaee10add4cd40259c97ad6f9b70d6c18500d3ad4a5f4a92683895f72497df7"}
Sep 11 17:48:45.215822 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:45.215763    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc" podStartSLOduration=1.034315141 podStartE2EDuration="1.215746912s" totalImagesPullingTime="181.431771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:48:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:48:45.215025819 +0000 UTC m=+593.103204166" watchObservedRunningTime="2026-09-11 17:48:45.215746912 +0000 UTC m=+593.103925294"
Sep 11 17:48:54.590053 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:54.590000    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:54.590053 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:54.590072    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:54.602713 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:54.602686    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:48:55.245359 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:48:55.245325    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-m9fwc"
Sep 11 17:49:21.382513 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.382427    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"]
Sep 11 17:49:21.386091 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.386055    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.389367 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.389341    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 11 17:49:21.399652 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.399626    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"]
Sep 11 17:49:21.464541 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.464504    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.464541 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.464542    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8327656a-ad02-4471-bd2a-1a5081be7e3b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.464775 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.464628    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jscjs\" (UniqueName: \"kubernetes.io/projected/8327656a-ad02-4471-bd2a-1a5081be7e3b-kube-api-access-jscjs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.464775 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.464732    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.464775 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.464759    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.464892 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.464800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.565710 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.565672    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.565710 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.565711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.565946 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.565743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.565946 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.565772    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.565946 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.565789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8327656a-ad02-4471-bd2a-1a5081be7e3b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.565946 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.565828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jscjs\" (UniqueName: \"kubernetes.io/projected/8327656a-ad02-4471-bd2a-1a5081be7e3b-kube-api-access-jscjs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.566194 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.566175    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.566250 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.566218    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.566333 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.566312    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.568228 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.568203    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8327656a-ad02-4471-bd2a-1a5081be7e3b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.568559 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.568534    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8327656a-ad02-4471-bd2a-1a5081be7e3b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.574717 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.574691    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jscjs\" (UniqueName: \"kubernetes.io/projected/8327656a-ad02-4471-bd2a-1a5081be7e3b-kube-api-access-jscjs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm\" (UID: \"8327656a-ad02-4471-bd2a-1a5081be7e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.695730 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.695689    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:21.969186 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.969162    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"]
Sep 11 17:49:21.971302 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:49:21.971270    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8327656a_ad02_4471_bd2a_1a5081be7e3b.slice/crio-28500a5a7227f9b0349de7b778b475f95ddd5af9429190df9925fbb43d11c00e WatchSource:0}: Error finding container 28500a5a7227f9b0349de7b778b475f95ddd5af9429190df9925fbb43d11c00e: Status 404 returned error can't find the container with id 28500a5a7227f9b0349de7b778b475f95ddd5af9429190df9925fbb43d11c00e
Sep 11 17:49:21.972619 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:21.972597    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:49:22.354311 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:22.354217    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm" event={"ID":"8327656a-ad02-4471-bd2a-1a5081be7e3b","Type":"ContainerStarted","Data":"8f117bff0db3eea4c0530122604c7a34a12e71562f429159d09f1a09f1fab411"}
Sep 11 17:49:22.354311 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:22.354267    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm" event={"ID":"8327656a-ad02-4471-bd2a-1a5081be7e3b","Type":"ContainerStarted","Data":"28500a5a7227f9b0349de7b778b475f95ddd5af9429190df9925fbb43d11c00e"}
Sep 11 17:49:22.375049 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:22.374989    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm" podStartSLOduration=1.199456623 podStartE2EDuration="1.374972545s" totalImagesPullingTime="175.515922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:49:22.373392221 +0000 UTC m=+630.261570566" watchObservedRunningTime="2026-09-11 17:49:22.374972545 +0000 UTC m=+630.263150888"
Sep 11 17:49:31.696140 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:31.696102    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:31.696624 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:31.696165    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:31.708953 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:31.708926    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:49:32.408604 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:49:32.408575    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm"
Sep 11 17:50:39.785784 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:39.785751    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d67687f77-p6pjc"]
Sep 11 17:50:39.789130 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:39.789111    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d67687f77-p6pjc"
Sep 11 17:50:39.797889 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:39.797860    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d67687f77-p6pjc"]
Sep 11 17:50:39.869823 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:39.869773    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d65f94f-e23d-4dbe-8a5a-397cf0339843-tls-cert\") pod \"authorino-d67687f77-p6pjc\" (UID: \"7d65f94f-e23d-4dbe-8a5a-397cf0339843\") " pod="kuadrant-system/authorino-d67687f77-p6pjc"
Sep 11 17:50:39.870047 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:39.869861    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db98w\" (UniqueName: \"kubernetes.io/projected/7d65f94f-e23d-4dbe-8a5a-397cf0339843-kube-api-access-db98w\") pod \"authorino-d67687f77-p6pjc\" (UID: \"7d65f94f-e23d-4dbe-8a5a-397cf0339843\") " pod="kuadrant-system/authorino-d67687f77-p6pjc"
Sep 11 17:50:39.970886 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:39.970850    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db98w\" (UniqueName: \"kubernetes.io/projected/7d65f94f-e23d-4dbe-8a5a-397cf0339843-kube-api-access-db98w\") pod \"authorino-d67687f77-p6pjc\" (UID: \"7d65f94f-e23d-4dbe-8a5a-397cf0339843\") " pod="kuadrant-system/authorino-d67687f77-p6pjc"
Sep 11 17:50:39.971064 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:39.970910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d65f94f-e23d-4dbe-8a5a-397cf0339843-tls-cert\") pod \"authorino-d67687f77-p6pjc\" (UID: \"7d65f94f-e23d-4dbe-8a5a-397cf0339843\") " pod="kuadrant-system/authorino-d67687f77-p6pjc"
Sep 11 17:50:39.973552 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:39.973531    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d65f94f-e23d-4dbe-8a5a-397cf0339843-tls-cert\") pod \"authorino-d67687f77-p6pjc\" (UID: \"7d65f94f-e23d-4dbe-8a5a-397cf0339843\") " pod="kuadrant-system/authorino-d67687f77-p6pjc"
Sep 11 17:50:39.980454 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:39.980427    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db98w\" (UniqueName: \"kubernetes.io/projected/7d65f94f-e23d-4dbe-8a5a-397cf0339843-kube-api-access-db98w\") pod \"authorino-d67687f77-p6pjc\" (UID: \"7d65f94f-e23d-4dbe-8a5a-397cf0339843\") " pod="kuadrant-system/authorino-d67687f77-p6pjc"
Sep 11 17:50:40.098880 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:40.098781    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d67687f77-p6pjc"
Sep 11 17:50:40.369879 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:40.369797    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d67687f77-p6pjc"]
Sep 11 17:50:40.373536 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:50:40.373500    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d65f94f_e23d_4dbe_8a5a_397cf0339843.slice/crio-cd2c0e02ef5f07bc48e6a26b059872abe6137025498033be13d52a201577a9bc WatchSource:0}: Error finding container cd2c0e02ef5f07bc48e6a26b059872abe6137025498033be13d52a201577a9bc: Status 404 returned error can't find the container with id cd2c0e02ef5f07bc48e6a26b059872abe6137025498033be13d52a201577a9bc
Sep 11 17:50:40.710514 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:40.710475    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d67687f77-p6pjc" event={"ID":"7d65f94f-e23d-4dbe-8a5a-397cf0339843","Type":"ContainerStarted","Data":"cd2c0e02ef5f07bc48e6a26b059872abe6137025498033be13d52a201577a9bc"}
Sep 11 17:50:41.719345 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:41.719306    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d67687f77-p6pjc" event={"ID":"7d65f94f-e23d-4dbe-8a5a-397cf0339843","Type":"ContainerStarted","Data":"7a99a54437933143f44e3331ef2d9c085aeae27d7b8ba08a48cc4bc422c2fdfe"}
Sep 11 17:50:41.737242 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:41.737174    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d67687f77-p6pjc" podStartSLOduration=2.019925078 podStartE2EDuration="2.737154244s" totalImagesPullingTime="717.229166ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:50:41.736459107 +0000 UTC m=+709.624637453" watchObservedRunningTime="2026-09-11 17:50:41.737154244 +0000 UTC m=+709.625332590"
Sep 11 17:50:41.768971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:41.768934    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66cd586c6-z2wcb"]
Sep 11 17:50:41.769167 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:41.769142    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66cd586c6-z2wcb" podUID="4b423686-c0b4-46ed-b316-e63ec050a57c" containerName="authorino" containerID="cri-o://c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa" gracePeriod=30
Sep 11 17:50:42.021656 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.021630    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66cd586c6-z2wcb"
Sep 11 17:50:42.089928 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.089893    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dktd\" (UniqueName: \"kubernetes.io/projected/4b423686-c0b4-46ed-b316-e63ec050a57c-kube-api-access-5dktd\") pod \"4b423686-c0b4-46ed-b316-e63ec050a57c\" (UID: \"4b423686-c0b4-46ed-b316-e63ec050a57c\") "
Sep 11 17:50:42.090140 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.089938    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b423686-c0b4-46ed-b316-e63ec050a57c-tls-cert\") pod \"4b423686-c0b4-46ed-b316-e63ec050a57c\" (UID: \"4b423686-c0b4-46ed-b316-e63ec050a57c\") "
Sep 11 17:50:42.092298 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.092259    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b423686-c0b4-46ed-b316-e63ec050a57c-kube-api-access-5dktd" (OuterVolumeSpecName: "kube-api-access-5dktd") pod "4b423686-c0b4-46ed-b316-e63ec050a57c" (UID: "4b423686-c0b4-46ed-b316-e63ec050a57c"). InnerVolumeSpecName "kube-api-access-5dktd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:50:42.099848 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.099811    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b423686-c0b4-46ed-b316-e63ec050a57c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4b423686-c0b4-46ed-b316-e63ec050a57c" (UID: "4b423686-c0b4-46ed-b316-e63ec050a57c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:50:42.191508 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.191461    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dktd\" (UniqueName: \"kubernetes.io/projected/4b423686-c0b4-46ed-b316-e63ec050a57c-kube-api-access-5dktd\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:50:42.191508 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.191494    2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4b423686-c0b4-46ed-b316-e63ec050a57c-tls-cert\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:50:42.726714 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.726673    2582 generic.go:356] "Generic (PLEG): container finished" podID="4b423686-c0b4-46ed-b316-e63ec050a57c" containerID="c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa" exitCode=0
Sep 11 17:50:42.727196 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.726726    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66cd586c6-z2wcb"
Sep 11 17:50:42.727196 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.726785    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66cd586c6-z2wcb" event={"ID":"4b423686-c0b4-46ed-b316-e63ec050a57c","Type":"ContainerDied","Data":"c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa"}
Sep 11 17:50:42.727196 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.726834    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66cd586c6-z2wcb" event={"ID":"4b423686-c0b4-46ed-b316-e63ec050a57c","Type":"ContainerDied","Data":"f4ea6c556e346d1d5f923e1e658c8bb263e891f738556b73e70d6018ccfd1cde"}
Sep 11 17:50:42.727196 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.726861    2582 scope.go:117] "RemoveContainer" containerID="c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa"
Sep 11 17:50:42.738063 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.738020    2582 scope.go:117] "RemoveContainer" containerID="c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa"
Sep 11 17:50:42.738330 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:50:42.738310    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa\": container with ID starting with c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa not found: ID does not exist" containerID="c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa"
Sep 11 17:50:42.738398 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.738357    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa"} err="failed to get container status \"c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa\": rpc error: code = NotFound desc = could not find container \"c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa\": container with ID starting with c892217357dea90ebe89dd49278cbc4c8eae85dcdb173d3d0c8608cccbecbdfa not found: ID does not exist"
Sep 11 17:50:42.745067 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.745043    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66cd586c6-z2wcb"]
Sep 11 17:50:42.748784 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:42.748756    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66cd586c6-z2wcb"]
Sep 11 17:50:44.633411 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:50:44.633336    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b423686-c0b4-46ed-b316-e63ec050a57c" path="/var/lib/kubelet/pods/4b423686-c0b4-46ed-b316-e63ec050a57c/volumes"
Sep 11 17:51:01.367494 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.367456    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8105-21a2c2"]
Sep 11 17:51:01.367876 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.367807    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b423686-c0b4-46ed-b316-e63ec050a57c" containerName="authorino"
Sep 11 17:51:01.367876 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.367820    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b423686-c0b4-46ed-b316-e63ec050a57c" containerName="authorino"
Sep 11 17:51:01.367945 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.367887    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b423686-c0b4-46ed-b316-e63ec050a57c" containerName="authorino"
Sep 11 17:51:01.372072 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.372051    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8105-21a2c2"
Sep 11 17:51:01.374952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.374927    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zsx9b\""
Sep 11 17:51:01.380164 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.380138    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8105-21a2c2"]
Sep 11 17:51:01.461067 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.461030    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff2r5\" (UniqueName: \"kubernetes.io/projected/c3cbbeea-e299-4c0a-9b79-073f1895d15e-kube-api-access-ff2r5\") pod \"test-curl-8105-21a2c2\" (UID: \"c3cbbeea-e299-4c0a-9b79-073f1895d15e\") " pod="opendatahub/test-curl-8105-21a2c2"
Sep 11 17:51:01.561936 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.561895    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff2r5\" (UniqueName: \"kubernetes.io/projected/c3cbbeea-e299-4c0a-9b79-073f1895d15e-kube-api-access-ff2r5\") pod \"test-curl-8105-21a2c2\" (UID: \"c3cbbeea-e299-4c0a-9b79-073f1895d15e\") " pod="opendatahub/test-curl-8105-21a2c2"
Sep 11 17:51:01.571987 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.571957    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff2r5\" (UniqueName: \"kubernetes.io/projected/c3cbbeea-e299-4c0a-9b79-073f1895d15e-kube-api-access-ff2r5\") pod \"test-curl-8105-21a2c2\" (UID: \"c3cbbeea-e299-4c0a-9b79-073f1895d15e\") " pod="opendatahub/test-curl-8105-21a2c2"
Sep 11 17:51:01.682023 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.681991    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8105-21a2c2"
Sep 11 17:51:01.957823 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:01.957799    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8105-21a2c2"]
Sep 11 17:51:01.962248 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:01.962213    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3cbbeea_e299_4c0a_9b79_073f1895d15e.slice/crio-2a867643493a39b3123659ab8fac270bac8f57cd0c1a7f9e92a423b045c2fc8a WatchSource:0}: Error finding container 2a867643493a39b3123659ab8fac270bac8f57cd0c1a7f9e92a423b045c2fc8a: Status 404 returned error can't find the container with id 2a867643493a39b3123659ab8fac270bac8f57cd0c1a7f9e92a423b045c2fc8a
Sep 11 17:51:02.818913 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:02.818871    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8105-21a2c2" event={"ID":"c3cbbeea-e299-4c0a-9b79-073f1895d15e","Type":"ContainerStarted","Data":"2a867643493a39b3123659ab8fac270bac8f57cd0c1a7f9e92a423b045c2fc8a"}
Sep 11 17:51:04.831488 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:04.831439    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8105-21a2c2" event={"ID":"c3cbbeea-e299-4c0a-9b79-073f1895d15e","Type":"ContainerStarted","Data":"9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a"}
Sep 11 17:51:04.848687 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:04.848625    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8105-21a2c2" podStartSLOduration=1.595932963 podStartE2EDuration="3.8486056s" totalImagesPullingTime="2.252672637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:04.847553388 +0000 UTC m=+732.735731737" watchObservedRunningTime="2026-09-11 17:51:04.8486056 +0000 UTC m=+732.736783946"
Sep 11 17:51:05.567805 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.567768    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8105-21a2c2"]
Sep 11 17:51:05.578753 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.578653    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8105-21a2c2"]
Sep 11 17:51:05.710569 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.710534    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8105-5c49ab"]
Sep 11 17:51:05.710952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.710934    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3cbbeea-e299-4c0a-9b79-073f1895d15e" containerName="test-curl-8105-21a2c2"
Sep 11 17:51:05.711037 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.710955    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3cbbeea-e299-4c0a-9b79-073f1895d15e" containerName="test-curl-8105-21a2c2"
Sep 11 17:51:05.711037 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.711030    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3cbbeea-e299-4c0a-9b79-073f1895d15e" containerName="test-curl-8105-21a2c2"
Sep 11 17:51:05.714056 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.714034    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8105-5c49ab"
Sep 11 17:51:05.724075 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.724048    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8105-5c49ab"]
Sep 11 17:51:05.803206 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.803171    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcnrz\" (UniqueName: \"kubernetes.io/projected/a013d5a1-1e58-46ae-852a-f99a3c265da0-kube-api-access-pcnrz\") pod \"test-curl-8105-5c49ab\" (UID: \"a013d5a1-1e58-46ae-852a-f99a3c265da0\") " pod="opendatahub/test-curl-8105-5c49ab"
Sep 11 17:51:05.836320 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:51:05.836237    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8105-21a2c2" container="test-curl-8105-21a2c2"
Sep 11 17:51:05.903951 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.903915    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcnrz\" (UniqueName: \"kubernetes.io/projected/a013d5a1-1e58-46ae-852a-f99a3c265da0-kube-api-access-pcnrz\") pod \"test-curl-8105-5c49ab\" (UID: \"a013d5a1-1e58-46ae-852a-f99a3c265da0\") " pod="opendatahub/test-curl-8105-5c49ab"
Sep 11 17:51:05.916270 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:05.916237    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcnrz\" (UniqueName: \"kubernetes.io/projected/a013d5a1-1e58-46ae-852a-f99a3c265da0-kube-api-access-pcnrz\") pod \"test-curl-8105-5c49ab\" (UID: \"a013d5a1-1e58-46ae-852a-f99a3c265da0\") " pod="opendatahub/test-curl-8105-5c49ab"
Sep 11 17:51:06.023611 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:06.023573    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8105-5c49ab"
Sep 11 17:51:06.299399 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:06.299367    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8105-5c49ab"]
Sep 11 17:51:06.302045 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:06.302017    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda013d5a1_1e58_46ae_852a_f99a3c265da0.slice/crio-fa45cbfba89b1cb367c5ea05a5c6d11b6aedb0f7e4bbfdb478333ea951d0cfed WatchSource:0}: Error finding container fa45cbfba89b1cb367c5ea05a5c6d11b6aedb0f7e4bbfdb478333ea951d0cfed: Status 404 returned error can't find the container with id fa45cbfba89b1cb367c5ea05a5c6d11b6aedb0f7e4bbfdb478333ea951d0cfed
Sep 11 17:51:06.842422 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:06.842385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8105-5c49ab" event={"ID":"a013d5a1-1e58-46ae-852a-f99a3c265da0","Type":"ContainerStarted","Data":"6bc1710a4ec3885cc2777ab70326a70988f5109a9603a2ce750811837b08382d"}
Sep 11 17:51:06.842422 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:06.842436    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8105-5c49ab" event={"ID":"a013d5a1-1e58-46ae-852a-f99a3c265da0","Type":"ContainerStarted","Data":"fa45cbfba89b1cb367c5ea05a5c6d11b6aedb0f7e4bbfdb478333ea951d0cfed"}
Sep 11 17:51:06.842920 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:06.842426    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8105-21a2c2" podUID="c3cbbeea-e299-4c0a-9b79-073f1895d15e" containerName="test-curl-8105-21a2c2" containerID="cri-o://9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a" gracePeriod=2
Sep 11 17:51:06.864596 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:06.864538    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8105-5c49ab" podStartSLOduration=1.864517523 podStartE2EDuration="1.864517523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:06.863195921 +0000 UTC m=+734.751374267" watchObservedRunningTime="2026-09-11 17:51:06.864517523 +0000 UTC m=+734.752695869"
Sep 11 17:51:07.638417 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:07.638326    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8105-5c49ab"]
Sep 11 17:51:07.644832 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:07.644804    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8105-5c49ab"]
Sep 11 17:51:07.847500 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:51:07.847469    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8105-5c49ab" container="test-curl-8105-5c49ab"
Sep 11 17:51:08.490859 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.490817    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"]
Sep 11 17:51:08.491203 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.491187    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a013d5a1-1e58-46ae-852a-f99a3c265da0" containerName="test-curl-8105-5c49ab"
Sep 11 17:51:08.491203 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.491203    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a013d5a1-1e58-46ae-852a-f99a3c265da0" containerName="test-curl-8105-5c49ab"
Sep 11 17:51:08.491342 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.491290    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a013d5a1-1e58-46ae-852a-f99a3c265da0" containerName="test-curl-8105-5c49ab"
Sep 11 17:51:08.494162 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.494139    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.497650 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.497626    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-37899e-openshift-default-dockercfg-b8blj\""
Sep 11 17:51:08.497970 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.497930    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 17:51:08.510872 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.510844    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"]
Sep 11 17:51:08.633110 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.633056    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-envoy\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.633278 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.633161    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-credential-socket\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.633278 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.633181    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-podinfo\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.633278 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.633197    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istiod-ca-cert\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.633394 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.633280    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96h7h\" (UniqueName: \"kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-kube-api-access-96h7h\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.633394 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.633305    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-socket\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.633394 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.633319    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-data\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.633394 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.633357    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-certs\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.633394 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.633382    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-token\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.734401 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.734361    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-certs\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.734401 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.734400    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-token\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.734632 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.734602    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-envoy\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.734793 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.734770    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-certs\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.735621 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.735069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-credential-socket\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.735621 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.735152    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-podinfo\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.735621 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.735195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istiod-ca-cert\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.735621 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.735376    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-credential-socket\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.735621 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.735576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96h7h\" (UniqueName: \"kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-kube-api-access-96h7h\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.735963 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.735651    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-socket\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.735963 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.735678    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-data\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.735963 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.735864    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istiod-ca-cert\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.736221 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.736202    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-data\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.736346 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.736323    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-socket\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.737309 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.737288    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-envoy\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.737748 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.737728    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-podinfo\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.744707 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.744634    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-token\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.745911 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.745891    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96h7h\" (UniqueName: \"kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-kube-api-access-96h7h\") pod \"e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") " pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.805798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.805756    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:08.853179 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:08.853149    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8105-5c49ab" podUID="a013d5a1-1e58-46ae-852a-f99a3c265da0" containerName="test-curl-8105-5c49ab" containerID="cri-o://6bc1710a4ec3885cc2777ab70326a70988f5109a9603a2ce750811837b08382d" gracePeriod=2
Sep 11 17:51:09.005214 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.005185    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8105-21a2c2"
Sep 11 17:51:09.093308 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.093273    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"]
Sep 11 17:51:09.096719 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:09.096691    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf56fc9f8_bfde_4e98_9b4c_cf04170b620d.slice/crio-b9fad1e7aa70612dda67c6f0a20523a7ed9385167d3fef990d4789e5a943638e WatchSource:0}: Error finding container b9fad1e7aa70612dda67c6f0a20523a7ed9385167d3fef990d4789e5a943638e: Status 404 returned error can't find the container with id b9fad1e7aa70612dda67c6f0a20523a7ed9385167d3fef990d4789e5a943638e
Sep 11 17:51:09.138404 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.138372    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff2r5\" (UniqueName: \"kubernetes.io/projected/c3cbbeea-e299-4c0a-9b79-073f1895d15e-kube-api-access-ff2r5\") pod \"c3cbbeea-e299-4c0a-9b79-073f1895d15e\" (UID: \"c3cbbeea-e299-4c0a-9b79-073f1895d15e\") "
Sep 11 17:51:09.140630 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.140605    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3cbbeea-e299-4c0a-9b79-073f1895d15e-kube-api-access-ff2r5" (OuterVolumeSpecName: "kube-api-access-ff2r5") pod "c3cbbeea-e299-4c0a-9b79-073f1895d15e" (UID: "c3cbbeea-e299-4c0a-9b79-073f1895d15e"). InnerVolumeSpecName "kube-api-access-ff2r5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:51:09.239965 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.239926    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff2r5\" (UniqueName: \"kubernetes.io/projected/c3cbbeea-e299-4c0a-9b79-073f1895d15e-kube-api-access-ff2r5\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:51:09.592729 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.592689    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"]
Sep 11 17:51:09.596136 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.596111    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.599596 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.599573    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-50ce3a65-openshift-default-dockercfg-4gpdq\""
Sep 11 17:51:09.611097 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.611048    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"]
Sep 11 17:51:09.744613 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.744575    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-token\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.744788 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.744631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-certs\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.744788 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.744662    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-credential-socket\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.744788 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.744689    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-data\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.744994 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.744784    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvdwr\" (UniqueName: \"kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-kube-api-access-hvdwr\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.744994 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.744817    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-podinfo\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.744994 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.744836    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-socket\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.744994 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.744875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istiod-ca-cert\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.744994 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.744908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-envoy\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.845971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.845887    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-token\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.845971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.845942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-certs\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.845977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-credential-socket\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.845999    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-data\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.846053    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvdwr\" (UniqueName: \"kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-kube-api-access-hvdwr\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.846097    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-podinfo\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846444 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.846301    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-socket\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846444 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.846358    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istiod-ca-cert\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846444 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.846391    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-envoy\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846444 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.846393    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-certs\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846644 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.846572    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-credential-socket\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846708 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.846648    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-socket\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.846763 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.846730    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-data\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.847358 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.847313    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istiod-ca-cert\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.849301 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.849279    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-envoy\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.849459 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.849434    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-podinfo\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.859299 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.858985    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvdwr\" (UniqueName: \"kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-kube-api-access-hvdwr\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.866046 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.864518    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-token\") pod \"e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") " pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:09.867745 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.867706    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" event={"ID":"f56fc9f8-bfde-4e98-9b4c-cf04170b620d","Type":"ContainerStarted","Data":"b9fad1e7aa70612dda67c6f0a20523a7ed9385167d3fef990d4789e5a943638e"}
Sep 11 17:51:09.869383 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.869355    2582 generic.go:356] "Generic (PLEG): container finished" podID="c3cbbeea-e299-4c0a-9b79-073f1895d15e" containerID="9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a" exitCode=137
Sep 11 17:51:09.869509 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.869472    2582 scope.go:117] "RemoveContainer" containerID="9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a"
Sep 11 17:51:09.869571 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.869503    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8105-21a2c2"
Sep 11 17:51:09.884692 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.884668    2582 scope.go:117] "RemoveContainer" containerID="9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a"
Sep 11 17:51:09.885067 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:51:09.885044    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a\": container with ID starting with 9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a not found: ID does not exist" containerID="9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a"
Sep 11 17:51:09.885198 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.885100    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a"} err="failed to get container status \"9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a\": rpc error: code = NotFound desc = could not find container \"9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a\": container with ID starting with 9fe47f534c3a86f38bc8c7127429c2a3e0bacc336ab9a08b927e03ab952af82a not found: ID does not exist"
Sep 11 17:51:09.885658 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:51:09.885633    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8105-21a2c2\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="c3cbbeea-e299-4c0a-9b79-073f1895d15e" pod="opendatahub/test-curl-8105-21a2c2"
Sep 11 17:51:09.905940 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:09.905904    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:10.195516 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.195390    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"]
Sep 11 17:51:10.197855 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:10.197818    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc9faa3_6d73_4861_b529_4bbdf03d219f.slice/crio-443251ea3085931102ae55d0c9f0f00ea5e9d7d4ac1948f44592b743a50c185a WatchSource:0}: Error finding container 443251ea3085931102ae55d0c9f0f00ea5e9d7d4ac1948f44592b743a50c185a: Status 404 returned error can't find the container with id 443251ea3085931102ae55d0c9f0f00ea5e9d7d4ac1948f44592b743a50c185a
Sep 11 17:51:10.448800 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.448723    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"]
Sep 11 17:51:10.453885 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.453857    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 17:51:10.457481 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:51:10.457440    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8105-21a2c2\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="c3cbbeea-e299-4c0a-9b79-073f1895d15e" pod="opendatahub/test-curl-8105-21a2c2"
Sep 11 17:51:10.457481 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.457475    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-4ab253-dockercfg-hjwt6\""
Sep 11 17:51:10.457641 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.457523    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-4ab253\""
Sep 11 17:51:10.460817 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.460771    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"]
Sep 11 17:51:10.554154 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.554107    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p986b\" (UniqueName: \"kubernetes.io/projected/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-kube-api-access-p986b\") pod \"payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l\" (UID: \"ab22c37f-c2ac-4247-bef9-48a84d2f02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 17:51:10.554328 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.554179    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l\" (UID: \"ab22c37f-c2ac-4247-bef9-48a84d2f02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 17:51:10.636178 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.636132    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3cbbeea-e299-4c0a-9b79-073f1895d15e" path="/var/lib/kubelet/pods/c3cbbeea-e299-4c0a-9b79-073f1895d15e/volumes"
Sep 11 17:51:10.655330 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.655291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p986b\" (UniqueName: \"kubernetes.io/projected/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-kube-api-access-p986b\") pod \"payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l\" (UID: \"ab22c37f-c2ac-4247-bef9-48a84d2f02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 17:51:10.655499 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.655354    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l\" (UID: \"ab22c37f-c2ac-4247-bef9-48a84d2f02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 17:51:10.656324 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.656298    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l\" (UID: \"ab22c37f-c2ac-4247-bef9-48a84d2f02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 17:51:10.670857 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.670823    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p986b\" (UniqueName: \"kubernetes.io/projected/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-kube-api-access-p986b\") pod \"payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l\" (UID: \"ab22c37f-c2ac-4247-bef9-48a84d2f02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 17:51:10.769305 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.769222    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 17:51:10.878828 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:10.878778    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" event={"ID":"1fc9faa3-6d73-4861-b529-4bbdf03d219f","Type":"ContainerStarted","Data":"443251ea3085931102ae55d0c9f0f00ea5e9d7d4ac1948f44592b743a50c185a"}
Sep 11 17:51:11.845694 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:11.845647    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:11.845773 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:11.845753    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:11.845822 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:11.845806    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:11.847748 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:11.847711    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:11.847826 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:11.847800    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:11.847876 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:11.847846    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:11.888070 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:11.888030    2582 generic.go:356] "Generic (PLEG): container finished" podID="a013d5a1-1e58-46ae-852a-f99a3c265da0" containerID="6bc1710a4ec3885cc2777ab70326a70988f5109a9603a2ce750811837b08382d" exitCode=137
Sep 11 17:51:11.911330 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:11.911294    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8105-5c49ab"
Sep 11 17:51:12.031515 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.031433    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"]
Sep 11 17:51:12.033947 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:12.033911    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab22c37f_c2ac_4247_bef9_48a84d2f02ca.slice/crio-b48c7c52f4a73dfb5115070d61d609f72e374fb1bd074251eb27e5f52c2d081d WatchSource:0}: Error finding container b48c7c52f4a73dfb5115070d61d609f72e374fb1bd074251eb27e5f52c2d081d: Status 404 returned error can't find the container with id b48c7c52f4a73dfb5115070d61d609f72e374fb1bd074251eb27e5f52c2d081d
Sep 11 17:51:12.068479 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.068158    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcnrz\" (UniqueName: \"kubernetes.io/projected/a013d5a1-1e58-46ae-852a-f99a3c265da0-kube-api-access-pcnrz\") pod \"a013d5a1-1e58-46ae-852a-f99a3c265da0\" (UID: \"a013d5a1-1e58-46ae-852a-f99a3c265da0\") "
Sep 11 17:51:12.070663 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.070630    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a013d5a1-1e58-46ae-852a-f99a3c265da0-kube-api-access-pcnrz" (OuterVolumeSpecName: "kube-api-access-pcnrz") pod "a013d5a1-1e58-46ae-852a-f99a3c265da0" (UID: "a013d5a1-1e58-46ae-852a-f99a3c265da0"). InnerVolumeSpecName "kube-api-access-pcnrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:51:12.169360 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.169320    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcnrz\" (UniqueName: \"kubernetes.io/projected/a013d5a1-1e58-46ae-852a-f99a3c265da0-kube-api-access-pcnrz\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:51:12.635789 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.635751    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a013d5a1-1e58-46ae-852a-f99a3c265da0" path="/var/lib/kubelet/pods/a013d5a1-1e58-46ae-852a-f99a3c265da0/volumes"
Sep 11 17:51:12.897236 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.897202    2582 scope.go:117] "RemoveContainer" containerID="6bc1710a4ec3885cc2777ab70326a70988f5109a9603a2ce750811837b08382d"
Sep 11 17:51:12.897699 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.897320    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8105-5c49ab"
Sep 11 17:51:12.900765 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.900728    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" event={"ID":"f56fc9f8-bfde-4e98-9b4c-cf04170b620d","Type":"ContainerStarted","Data":"c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f"}
Sep 11 17:51:12.904288 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.904242    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" event={"ID":"1fc9faa3-6d73-4861-b529-4bbdf03d219f","Type":"ContainerStarted","Data":"fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43"}
Sep 11 17:51:12.906213 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.906052    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:12.908432 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.908098    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l" event={"ID":"ab22c37f-c2ac-4247-bef9-48a84d2f02ca","Type":"ContainerStarted","Data":"b48c7c52f4a73dfb5115070d61d609f72e374fb1bd074251eb27e5f52c2d081d"}
Sep 11 17:51:12.913124 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.913103    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:12.954721 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.954647    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" podStartSLOduration=2.307656611 podStartE2EDuration="3.954625891s" totalImagesPullingTime="1.64696928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:12.95310687 +0000 UTC m=+740.841285220" watchObservedRunningTime="2026-09-11 17:51:12.954625891 +0000 UTC m=+740.842804262"
Sep 11 17:51:12.955953 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:12.955915    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" podStartSLOduration=2.209713925 podStartE2EDuration="4.955902689s" totalImagesPullingTime="2.746188764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:12.927549941 +0000 UTC m=+740.815728292" watchObservedRunningTime="2026-09-11 17:51:12.955902689 +0000 UTC m=+740.844081067"
Sep 11 17:51:13.806258 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:13.806222    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:13.811858 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:13.811823    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:13.915899 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:13.915770    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:13.915899 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:13.915838    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:13.916683 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:13.916659    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:13.916994 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:13.916976    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:14.923910 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:14.923856    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l" event={"ID":"ab22c37f-c2ac-4247-bef9-48a84d2f02ca","Type":"ContainerStarted","Data":"f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d"}
Sep 11 17:51:14.924460 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:14.924392    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 17:51:18.650941 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.650885    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l" podStartSLOduration=6.2067805289999995 podStartE2EDuration="8.650864842s" totalImagesPullingTime="2.444084313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:14.961778208 +0000 UTC m=+742.849956554" watchObservedRunningTime="2026-09-11 17:51:18.650864842 +0000 UTC m=+746.539043187"
Sep 11 17:51:18.651438 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.651307    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"]
Sep 11 17:51:18.654362 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.654338    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.657151 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.657129    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5vrxn\""
Sep 11 17:51:18.658375 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.658341    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 17:51:18.659356 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.659338    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:51:18.659454 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.659396    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 17:51:18.659454 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.659439    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-37899e-serving-cert\""
Sep 11 17:51:18.671288 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.671254    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"]
Sep 11 17:51:18.731816 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.731777    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-maas-api-tls\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.731816 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.731815    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fqv4\" (UniqueName: \"kubernetes.io/projected/68c857a9-3cab-47a1-a15a-be0482ac0c87-kube-api-access-9fqv4\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.732124 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.731854    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68c857a9-3cab-47a1-a15a-be0482ac0c87-openshift-service-ca\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.732124 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.731881    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-metrics-certs\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.833389 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.833348    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-maas-api-tls\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.833389 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.833392    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fqv4\" (UniqueName: \"kubernetes.io/projected/68c857a9-3cab-47a1-a15a-be0482ac0c87-kube-api-access-9fqv4\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.833647 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.833428    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68c857a9-3cab-47a1-a15a-be0482ac0c87-openshift-service-ca\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.833647 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.833445    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-metrics-certs\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.833647 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:51:18.833540    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-37899e-serving-cert: secret "maas-api-e2e-autores-37899e-serving-cert" not found
Sep 11 17:51:18.833647 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:51:18.833629    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-maas-api-tls podName:68c857a9-3cab-47a1-a15a-be0482ac0c87 nodeName:}" failed. No retries permitted until 2026-09-11 17:51:19.333605986 +0000 UTC m=+747.221784334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-maas-api-tls") pod "maas-api-e2e-autores-37899e-85977bfc46-788ph" (UID: "68c857a9-3cab-47a1-a15a-be0482ac0c87") : secret "maas-api-e2e-autores-37899e-serving-cert" not found
Sep 11 17:51:18.834074 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.834047    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68c857a9-3cab-47a1-a15a-be0482ac0c87-openshift-service-ca\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.836028 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.836007    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-metrics-certs\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:18.843702 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:18.843673    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fqv4\" (UniqueName: \"kubernetes.io/projected/68c857a9-3cab-47a1-a15a-be0482ac0c87-kube-api-access-9fqv4\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:19.338536 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.338489    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-maas-api-tls\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:19.341144 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.341121    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-maas-api-tls\") pod \"maas-api-e2e-autores-37899e-85977bfc46-788ph\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:19.567738 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.567702    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:19.612436 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.612362    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"]
Sep 11 17:51:19.615832 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.615810    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:51:19.620113 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.620033    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-37899e\""
Sep 11 17:51:19.620837 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.620648    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-37899e-dockercfg-v8rbj\""
Sep 11 17:51:19.632188 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.632156    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"]
Sep 11 17:51:19.641026 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.640978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42814990-b4d0-47a7-a687-ef0a597e7f3d-plugins-config-volume\") pod \"payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b\" (UID: \"42814990-b4d0-47a7-a687-ef0a597e7f3d\") " pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:51:19.641496 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.641177    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzwz2\" (UniqueName: \"kubernetes.io/projected/42814990-b4d0-47a7-a687-ef0a597e7f3d-kube-api-access-vzwz2\") pod \"payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b\" (UID: \"42814990-b4d0-47a7-a687-ef0a597e7f3d\") " pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:51:19.742460 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.742423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzwz2\" (UniqueName: \"kubernetes.io/projected/42814990-b4d0-47a7-a687-ef0a597e7f3d-kube-api-access-vzwz2\") pod \"payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b\" (UID: \"42814990-b4d0-47a7-a687-ef0a597e7f3d\") " pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:51:19.742928 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.742474    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42814990-b4d0-47a7-a687-ef0a597e7f3d-plugins-config-volume\") pod \"payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b\" (UID: \"42814990-b4d0-47a7-a687-ef0a597e7f3d\") " pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:51:19.743338 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.743315    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42814990-b4d0-47a7-a687-ef0a597e7f3d-plugins-config-volume\") pod \"payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b\" (UID: \"42814990-b4d0-47a7-a687-ef0a597e7f3d\") " pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:51:19.751784 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.751754    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzwz2\" (UniqueName: \"kubernetes.io/projected/42814990-b4d0-47a7-a687-ef0a597e7f3d-kube-api-access-vzwz2\") pod \"payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b\" (UID: \"42814990-b4d0-47a7-a687-ef0a597e7f3d\") " pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:51:19.846233 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.846204    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"]
Sep 11 17:51:19.848265 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:19.848237    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68c857a9_3cab_47a1_a15a_be0482ac0c87.slice/crio-61f679ea648ac354825a5fc59481a6c6b00e8d3eb20d7a3f9cfbe35e81ebadc1 WatchSource:0}: Error finding container 61f679ea648ac354825a5fc59481a6c6b00e8d3eb20d7a3f9cfbe35e81ebadc1: Status 404 returned error can't find the container with id 61f679ea648ac354825a5fc59481a6c6b00e8d3eb20d7a3f9cfbe35e81ebadc1
Sep 11 17:51:19.934859 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.934813    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:51:19.951214 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:19.951178    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph" event={"ID":"68c857a9-3cab-47a1-a15a-be0482ac0c87","Type":"ContainerStarted","Data":"61f679ea648ac354825a5fc59481a6c6b00e8d3eb20d7a3f9cfbe35e81ebadc1"}
Sep 11 17:51:20.226044 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:20.226014    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"]
Sep 11 17:51:20.228710 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:20.228676    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42814990_b4d0_47a7_a687_ef0a597e7f3d.slice/crio-427c824ceb643181dff3a9adf4282c979535f0e7a23669a09569149a83091bc4 WatchSource:0}: Error finding container 427c824ceb643181dff3a9adf4282c979535f0e7a23669a09569149a83091bc4: Status 404 returned error can't find the container with id 427c824ceb643181dff3a9adf4282c979535f0e7a23669a09569149a83091bc4
Sep 11 17:51:20.960501 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:20.960448    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b" event={"ID":"42814990-b4d0-47a7-a687-ef0a597e7f3d","Type":"ContainerStarted","Data":"79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448"}
Sep 11 17:51:20.960988 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:20.960514    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b" event={"ID":"42814990-b4d0-47a7-a687-ef0a597e7f3d","Type":"ContainerStarted","Data":"427c824ceb643181dff3a9adf4282c979535f0e7a23669a09569149a83091bc4"}
Sep 11 17:51:20.960988 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:20.960543    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:51:20.989417 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:20.989385    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"]
Sep 11 17:51:20.993908 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:20.993416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:20.997797 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:20.997770    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-50ce3a65\""
Sep 11 17:51:20.997797 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:20.997795    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-50ce3a65-dockercfg-2hmfj\""
Sep 11 17:51:21.010972 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:21.010911    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b" podStartSLOduration=2.010886895 podStartE2EDuration="2.010886895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:21.002635241 +0000 UTC m=+748.890813584" watchObservedRunningTime="2026-09-11 17:51:21.010886895 +0000 UTC m=+748.899065241"
Sep 11 17:51:21.024004 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:21.023965    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"]
Sep 11 17:51:21.056657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:21.056211    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f0e156-ffb2-41e1-9ec1-7295423e24fa-plugins-config-volume\") pod \"payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl\" (UID: \"11f0e156-ffb2-41e1-9ec1-7295423e24fa\") " pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:21.056657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:21.056277    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzjq2\" (UniqueName: \"kubernetes.io/projected/11f0e156-ffb2-41e1-9ec1-7295423e24fa-kube-api-access-hzjq2\") pod \"payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl\" (UID: \"11f0e156-ffb2-41e1-9ec1-7295423e24fa\") " pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:21.157103 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:21.156936    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f0e156-ffb2-41e1-9ec1-7295423e24fa-plugins-config-volume\") pod \"payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl\" (UID: \"11f0e156-ffb2-41e1-9ec1-7295423e24fa\") " pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:21.157103 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:21.157000    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzjq2\" (UniqueName: \"kubernetes.io/projected/11f0e156-ffb2-41e1-9ec1-7295423e24fa-kube-api-access-hzjq2\") pod \"payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl\" (UID: \"11f0e156-ffb2-41e1-9ec1-7295423e24fa\") " pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:21.158521 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:21.158334    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f0e156-ffb2-41e1-9ec1-7295423e24fa-plugins-config-volume\") pod \"payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl\" (UID: \"11f0e156-ffb2-41e1-9ec1-7295423e24fa\") " pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:21.171771 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:21.171736    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzjq2\" (UniqueName: \"kubernetes.io/projected/11f0e156-ffb2-41e1-9ec1-7295423e24fa-kube-api-access-hzjq2\") pod \"payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl\" (UID: \"11f0e156-ffb2-41e1-9ec1-7295423e24fa\") " pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:21.310870 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:21.310775    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:22.678961 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:22.678936    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"]
Sep 11 17:51:22.680868 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:22.680839    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f0e156_ffb2_41e1_9ec1_7295423e24fa.slice/crio-f85614d0cd0da6c8eeaa73a08a6f5fe65a8304757d268c42059a66a13ece0802 WatchSource:0}: Error finding container f85614d0cd0da6c8eeaa73a08a6f5fe65a8304757d268c42059a66a13ece0802: Status 404 returned error can't find the container with id f85614d0cd0da6c8eeaa73a08a6f5fe65a8304757d268c42059a66a13ece0802
Sep 11 17:51:22.972477 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:22.972369    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl" event={"ID":"11f0e156-ffb2-41e1-9ec1-7295423e24fa","Type":"ContainerStarted","Data":"90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f"}
Sep 11 17:51:22.972477 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:22.972465    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl" event={"ID":"11f0e156-ffb2-41e1-9ec1-7295423e24fa","Type":"ContainerStarted","Data":"f85614d0cd0da6c8eeaa73a08a6f5fe65a8304757d268c42059a66a13ece0802"}
Sep 11 17:51:22.972695 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:22.972498    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:22.974019 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:22.973994    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph" event={"ID":"68c857a9-3cab-47a1-a15a-be0482ac0c87","Type":"ContainerStarted","Data":"ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351"}
Sep 11 17:51:22.974170 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:22.974136    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:22.994861 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:22.994805    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl" podStartSLOduration=2.994785378 podStartE2EDuration="2.994785378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:22.994132592 +0000 UTC m=+750.882310941" watchObservedRunningTime="2026-09-11 17:51:22.994785378 +0000 UTC m=+750.882963730"
Sep 11 17:51:23.014148 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:23.014095    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph" podStartSLOduration=2.399150692 podStartE2EDuration="5.01406102s" totalImagesPullingTime="2.614910328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:23.012900056 +0000 UTC m=+750.901078403" watchObservedRunningTime="2026-09-11 17:51:23.01406102 +0000 UTC m=+750.902239417"
Sep 11 17:51:26.935819 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:26.935783    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 17:51:28.987317 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:28.987290    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:51:29.630887 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.630847    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"]
Sep 11 17:51:29.634669 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.634647    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:51:29.635902 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.635878    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"]
Sep 11 17:51:29.637476 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.637456    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-9aab53-dockercfg-hmfxv\""
Sep 11 17:51:29.637788 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.637768    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-9aab53\""
Sep 11 17:51:29.639989 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.639969    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:51:29.645800 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.645774    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"]
Sep 11 17:51:29.652822 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.652795    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"]
Sep 11 17:51:29.743751 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.743715    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqrx7\" (UniqueName: \"kubernetes.io/projected/e45c7d90-67ec-490c-bb93-d18c0d6a0616-kube-api-access-wqrx7\") pod \"payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg\" (UID: \"e45c7d90-67ec-490c-bb93-d18c0d6a0616\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:51:29.743926 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.743876    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e45c7d90-67ec-490c-bb93-d18c0d6a0616-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg\" (UID: \"e45c7d90-67ec-490c-bb93-d18c0d6a0616\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:51:29.743990 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.743923    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd1bdafc-7907-4eca-8299-265c35f1930f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr\" (UID: \"bd1bdafc-7907-4eca-8299-265c35f1930f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:51:29.743990 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.743952    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt4zp\" (UniqueName: \"kubernetes.io/projected/bd1bdafc-7907-4eca-8299-265c35f1930f-kube-api-access-bt4zp\") pod \"payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr\" (UID: \"bd1bdafc-7907-4eca-8299-265c35f1930f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:51:29.844931 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.844893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqrx7\" (UniqueName: \"kubernetes.io/projected/e45c7d90-67ec-490c-bb93-d18c0d6a0616-kube-api-access-wqrx7\") pod \"payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg\" (UID: \"e45c7d90-67ec-490c-bb93-d18c0d6a0616\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:51:29.845142 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.844999    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e45c7d90-67ec-490c-bb93-d18c0d6a0616-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg\" (UID: \"e45c7d90-67ec-490c-bb93-d18c0d6a0616\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:51:29.845142 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.845023    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd1bdafc-7907-4eca-8299-265c35f1930f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr\" (UID: \"bd1bdafc-7907-4eca-8299-265c35f1930f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:51:29.845142 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.845040    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt4zp\" (UniqueName: \"kubernetes.io/projected/bd1bdafc-7907-4eca-8299-265c35f1930f-kube-api-access-bt4zp\") pod \"payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr\" (UID: \"bd1bdafc-7907-4eca-8299-265c35f1930f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:51:29.845811 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.845789    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd1bdafc-7907-4eca-8299-265c35f1930f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr\" (UID: \"bd1bdafc-7907-4eca-8299-265c35f1930f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:51:29.845958 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.845934    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e45c7d90-67ec-490c-bb93-d18c0d6a0616-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg\" (UID: \"e45c7d90-67ec-490c-bb93-d18c0d6a0616\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:51:29.855036 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.855008    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqrx7\" (UniqueName: \"kubernetes.io/projected/e45c7d90-67ec-490c-bb93-d18c0d6a0616-kube-api-access-wqrx7\") pod \"payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg\" (UID: \"e45c7d90-67ec-490c-bb93-d18c0d6a0616\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:51:29.855177 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.855008    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt4zp\" (UniqueName: \"kubernetes.io/projected/bd1bdafc-7907-4eca-8299-265c35f1930f-kube-api-access-bt4zp\") pod \"payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr\" (UID: \"bd1bdafc-7907-4eca-8299-265c35f1930f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:51:29.952258 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.952218    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:51:29.960464 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:29.960424    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:51:30.242787 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:30.242753    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"]
Sep 11 17:51:30.245106 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:30.245053    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45c7d90_67ec_490c_bb93_d18c0d6a0616.slice/crio-f8bd780f16dae392a99a15e5a30627295dc332828ee611b77e0cc25c28d21b51 WatchSource:0}: Error finding container f8bd780f16dae392a99a15e5a30627295dc332828ee611b77e0cc25c28d21b51: Status 404 returned error can't find the container with id f8bd780f16dae392a99a15e5a30627295dc332828ee611b77e0cc25c28d21b51
Sep 11 17:51:30.245549 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:30.245528    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"]
Sep 11 17:51:30.247607 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:30.247586    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1bdafc_7907_4eca_8299_265c35f1930f.slice/crio-ba6db720ad9792fe5e18237055d8bbc3ea83d298905fe003b1c9862c4c58138e WatchSource:0}: Error finding container ba6db720ad9792fe5e18237055d8bbc3ea83d298905fe003b1c9862c4c58138e: Status 404 returned error can't find the container with id ba6db720ad9792fe5e18237055d8bbc3ea83d298905fe003b1c9862c4c58138e
Sep 11 17:51:31.019754 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:31.019712    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg" event={"ID":"e45c7d90-67ec-490c-bb93-d18c0d6a0616","Type":"ContainerStarted","Data":"58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de"}
Sep 11 17:51:31.019949 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:31.019780    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg" event={"ID":"e45c7d90-67ec-490c-bb93-d18c0d6a0616","Type":"ContainerStarted","Data":"f8bd780f16dae392a99a15e5a30627295dc332828ee611b77e0cc25c28d21b51"}
Sep 11 17:51:31.019949 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:31.019924    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:51:31.021685 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:31.021656    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr" event={"ID":"bd1bdafc-7907-4eca-8299-265c35f1930f","Type":"ContainerStarted","Data":"976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67"}
Sep 11 17:51:31.021813 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:31.021712    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr" event={"ID":"bd1bdafc-7907-4eca-8299-265c35f1930f","Type":"ContainerStarted","Data":"ba6db720ad9792fe5e18237055d8bbc3ea83d298905fe003b1c9862c4c58138e"}
Sep 11 17:51:31.021813 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:31.021742    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:51:31.044921 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:31.044853    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg" podStartSLOduration=2.044832406 podStartE2EDuration="2.044832406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:31.042729875 +0000 UTC m=+758.930908238" watchObservedRunningTime="2026-09-11 17:51:31.044832406 +0000 UTC m=+758.933010752"
Sep 11 17:51:31.062642 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:31.062588    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr" podStartSLOduration=2.062569619 podStartE2EDuration="2.062569619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:31.060613411 +0000 UTC m=+758.948791758" watchObservedRunningTime="2026-09-11 17:51:31.062569619 +0000 UTC m=+758.950747964"
Sep 11 17:51:31.967734 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:31.967702    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:51:33.984336 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:33.984255    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:41.107852 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:41.107812    2582 generic.go:356] "Generic (PLEG): container finished" podID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerID="fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43" exitCode=0
Sep 11 17:51:41.108324 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:41.107898    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" event={"ID":"1fc9faa3-6d73-4861-b529-4bbdf03d219f","Type":"ContainerDied","Data":"fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43"}
Sep 11 17:51:41.108552 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:41.108530    2582 scope.go:117] "RemoveContainer" containerID="fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43"
Sep 11 17:51:41.109957 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:41.109916    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:41.110035 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:41.110016    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:41.110113 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:41.110069    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:42.043618 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:42.043266    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:51:42.044050 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:42.044026    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:51:42.125539 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:42.125493    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" event={"ID":"1fc9faa3-6d73-4861-b529-4bbdf03d219f","Type":"ContainerStarted","Data":"30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef"}
Sep 11 17:51:42.906440 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:42.906396    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:42.908172 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:42.907614    2582 patch_prober.go:28] interesting pod/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 11 17:51:42.908172 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:42.907664    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 11 17:51:43.137232 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:43.135050    2582 generic.go:356] "Generic (PLEG): container finished" podID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerID="c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f" exitCode=0
Sep 11 17:51:43.137232 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:43.135966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" event={"ID":"f56fc9f8-bfde-4e98-9b4c-cf04170b620d","Type":"ContainerDied","Data":"c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f"}
Sep 11 17:51:43.137232 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:43.136464    2582 scope.go:117] "RemoveContainer" containerID="c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f"
Sep 11 17:51:43.139973 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:43.139623    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:43.139973 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:43.139724    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:43.139973 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:43.139774    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:51:43.906891 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:43.906855    2582 patch_prober.go:28] interesting pod/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 11 17:51:43.907075 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:43.906912    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 11 17:51:43.915926 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:43.915836    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:43.916236 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:43.916216    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:44.151570 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:44.151527    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" event={"ID":"f56fc9f8-bfde-4e98-9b4c-cf04170b620d","Type":"ContainerStarted","Data":"0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea"}
Sep 11 17:51:44.806479 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:44.806429    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:44.807716 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:44.807692    2582 patch_prober.go:28] interesting pod/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Sep 11 17:51:44.807897 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:44.807871    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Sep 11 17:51:44.906807 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:44.906761    2582 patch_prober.go:28] interesting pod/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 11 17:51:44.907012 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:44.906822    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 11 17:51:45.806575 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:45.806533    2582 patch_prober.go:28] interesting pod/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Sep 11 17:51:45.806979 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:45.806591    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Sep 11 17:51:45.907165 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:45.907127    2582 patch_prober.go:28] interesting pod/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 11 17:51:45.907346 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:45.907187    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 11 17:51:46.806523 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:46.806483    2582 patch_prober.go:28] interesting pod/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Sep 11 17:51:46.806717 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:46.806540    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Sep 11 17:51:46.907061 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:46.907020    2582 patch_prober.go:28] interesting pod/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 11 17:51:46.907249 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:46.907076    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 11 17:51:47.806862 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:47.806821    2582 patch_prober.go:28] interesting pod/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Sep 11 17:51:47.807306 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:47.806880    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Sep 11 17:51:47.912675 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:47.912638    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:47.913909 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:47.913886    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:51:48.806227 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:48.806192    2582 patch_prober.go:28] interesting pod/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Sep 11 17:51:48.806413 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:48.806247    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Sep 11 17:51:49.811395 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:49.811361    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:49.811797 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:49.811652    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:49.812409 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:49.812393    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:51:56.530715 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:56.529556    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"]
Sep 11 17:51:56.530715 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:56.529879    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl" podUID="11f0e156-ffb2-41e1-9ec1-7295423e24fa" containerName="payload-processing" containerID="cri-o://90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f" gracePeriod=30
Sep 11 17:51:56.789660 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:56.789587    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:56.874217 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:56.874178    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"]
Sep 11 17:51:56.874477 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:56.874449    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy" containerID="cri-o://30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef" gracePeriod=30
Sep 11 17:51:56.932049 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:56.932011    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzjq2\" (UniqueName: \"kubernetes.io/projected/11f0e156-ffb2-41e1-9ec1-7295423e24fa-kube-api-access-hzjq2\") pod \"11f0e156-ffb2-41e1-9ec1-7295423e24fa\" (UID: \"11f0e156-ffb2-41e1-9ec1-7295423e24fa\") "
Sep 11 17:51:56.932270 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:56.932155    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f0e156-ffb2-41e1-9ec1-7295423e24fa-plugins-config-volume\") pod \"11f0e156-ffb2-41e1-9ec1-7295423e24fa\" (UID: \"11f0e156-ffb2-41e1-9ec1-7295423e24fa\") "
Sep 11 17:51:56.932550 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:56.932524    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11f0e156-ffb2-41e1-9ec1-7295423e24fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11f0e156-ffb2-41e1-9ec1-7295423e24fa" (UID: "11f0e156-ffb2-41e1-9ec1-7295423e24fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:51:56.934649 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:56.934620    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f0e156-ffb2-41e1-9ec1-7295423e24fa-kube-api-access-hzjq2" (OuterVolumeSpecName: "kube-api-access-hzjq2") pod "11f0e156-ffb2-41e1-9ec1-7295423e24fa" (UID: "11f0e156-ffb2-41e1-9ec1-7295423e24fa"). InnerVolumeSpecName "kube-api-access-hzjq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:51:57.033695 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.033650    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f0e156-ffb2-41e1-9ec1-7295423e24fa-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:51:57.033695 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.033686    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzjq2\" (UniqueName: \"kubernetes.io/projected/11f0e156-ffb2-41e1-9ec1-7295423e24fa-kube-api-access-hzjq2\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:51:57.222903 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.222866    2582 generic.go:356] "Generic (PLEG): container finished" podID="11f0e156-ffb2-41e1-9ec1-7295423e24fa" containerID="90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f" exitCode=0
Sep 11 17:51:57.223113 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.222912    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"
Sep 11 17:51:57.223113 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.222921    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl" event={"ID":"11f0e156-ffb2-41e1-9ec1-7295423e24fa","Type":"ContainerDied","Data":"90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f"}
Sep 11 17:51:57.223113 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.222975    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl" event={"ID":"11f0e156-ffb2-41e1-9ec1-7295423e24fa","Type":"ContainerDied","Data":"f85614d0cd0da6c8eeaa73a08a6f5fe65a8304757d268c42059a66a13ece0802"}
Sep 11 17:51:57.223113 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.223003    2582 scope.go:117] "RemoveContainer" containerID="90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f"
Sep 11 17:51:57.232280 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.232258    2582 scope.go:117] "RemoveContainer" containerID="90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f"
Sep 11 17:51:57.232584 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:51:57.232564    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f\": container with ID starting with 90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f not found: ID does not exist" containerID="90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f"
Sep 11 17:51:57.232653 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.232594    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f"} err="failed to get container status \"90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f\": rpc error: code = NotFound desc = could not find container \"90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f\": container with ID starting with 90e689a0edec140efdbb4a5d13a791e7fe121edfca3282271ea678b0792b799f not found: ID does not exist"
Sep 11 17:51:57.255501 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.255463    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"]
Sep 11 17:51:57.279714 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:57.279674    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-50ce3a65-67477bb875-w9hzl"]
Sep 11 17:51:58.165213 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.165170    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"]
Sep 11 17:51:58.165795 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.165766    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11f0e156-ffb2-41e1-9ec1-7295423e24fa" containerName="payload-processing"
Sep 11 17:51:58.165795 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.165795    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="11f0e156-ffb2-41e1-9ec1-7295423e24fa" containerName="payload-processing"
Sep 11 17:51:58.165989 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.165909    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="11f0e156-ffb2-41e1-9ec1-7295423e24fa" containerName="payload-processing"
Sep 11 17:51:58.170001 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.169980    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.174117 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.174074    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"openshift-service-ca.crt\""
Sep 11 17:51:58.174117 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.174102    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"simulator-w0-286c27-kserve-self-signed-certs\""
Sep 11 17:51:58.174322 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.174129    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"default-dockercfg-n6lmt\""
Sep 11 17:51:58.174322 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.174159    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"kube-root-ca.crt\""
Sep 11 17:51:58.178449 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.178428    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"]
Sep 11 17:51:58.346159 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.346075    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-home\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.346333 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.346207    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tmp-dir\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.346333 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.346276    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-dshm\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.346333 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.346304    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-model-cache\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.346333 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.346329    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tls-certs\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.346480 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.346350    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tm6p\" (UniqueName: \"kubernetes.io/projected/78f62f94-9e42-4bca-bd2c-898cd8d1c807-kube-api-access-2tm6p\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.447247 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.447146    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tls-certs\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.447247 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.447196    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tm6p\" (UniqueName: \"kubernetes.io/projected/78f62f94-9e42-4bca-bd2c-898cd8d1c807-kube-api-access-2tm6p\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.447488 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.447279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-home\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.447488 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.447328    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tmp-dir\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.447488 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.447362    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-dshm\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.447488 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.447384    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-model-cache\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.447740 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.447721    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-home\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.447795 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.447755    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-model-cache\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.447841 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.447817    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tmp-dir\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.449814 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.449785    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-dshm\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.449935 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.449919    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tls-certs\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.457580 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.457554    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tm6p\" (UniqueName: \"kubernetes.io/projected/78f62f94-9e42-4bca-bd2c-898cd8d1c807-kube-api-access-2tm6p\") pod \"simulator-w0-286c27-kserve-6bc9f584dc-zz5zm\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") " pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.480479 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.480435    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:51:58.634534 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.634499    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11f0e156-ffb2-41e1-9ec1-7295423e24fa" path="/var/lib/kubelet/pods/11f0e156-ffb2-41e1-9ec1-7295423e24fa/volumes"
Sep 11 17:51:58.764705 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:58.764677    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"]
Sep 11 17:51:58.766970 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:51:58.766943    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78f62f94_9e42_4bca_bd2c_898cd8d1c807.slice/crio-1d2b9e5ef7c5353ee9bb509869df28dfd56b62d38b6169bd8e913d7c7b5b683c WatchSource:0}: Error finding container 1d2b9e5ef7c5353ee9bb509869df28dfd56b62d38b6169bd8e913d7c7b5b683c: Status 404 returned error can't find the container with id 1d2b9e5ef7c5353ee9bb509869df28dfd56b62d38b6169bd8e913d7c7b5b683c
Sep 11 17:51:59.237010 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:59.236963    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm" event={"ID":"78f62f94-9e42-4bca-bd2c-898cd8d1c807","Type":"ContainerStarted","Data":"0f7bff50d2ed0356d83571c0ef05aef25b3af2845cc9c97e387a82a6f2b52731"}
Sep 11 17:51:59.237010 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:59.237012    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm" event={"ID":"78f62f94-9e42-4bca-bd2c-898cd8d1c807","Type":"ContainerStarted","Data":"1d2b9e5ef7c5353ee9bb509869df28dfd56b62d38b6169bd8e913d7c7b5b683c"}
Sep 11 17:51:59.259924 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:51:59.259863    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm" podStartSLOduration=1.259845528 podStartE2EDuration="1.259845528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:59.258659151 +0000 UTC m=+787.146837501" watchObservedRunningTime="2026-09-11 17:51:59.259845528 +0000 UTC m=+787.148023873"
Sep 11 17:52:02.136470 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.136400    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:52:02.185638 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185598    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-certs\") pod \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") "
Sep 11 17:52:02.185638 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185643    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-podinfo\") pod \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") "
Sep 11 17:52:02.185885 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185672    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-token\") pod \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") "
Sep 11 17:52:02.185885 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185697    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-credential-socket\") pod \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") "
Sep 11 17:52:02.185885 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185723    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-envoy\") pod \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") "
Sep 11 17:52:02.185885 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185757    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-socket\") pod \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") "
Sep 11 17:52:02.185885 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185785    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-data\") pod \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") "
Sep 11 17:52:02.185885 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185853    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvdwr\" (UniqueName: \"kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-kube-api-access-hvdwr\") pod \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") "
Sep 11 17:52:02.186226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185928    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istiod-ca-cert\") pod \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\" (UID: \"1fc9faa3-6d73-4861-b529-4bbdf03d219f\") "
Sep 11 17:52:02.186226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185960    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1fc9faa3-6d73-4861-b529-4bbdf03d219f" (UID: "1fc9faa3-6d73-4861-b529-4bbdf03d219f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:52:02.186226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.185972    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1fc9faa3-6d73-4861-b529-4bbdf03d219f" (UID: "1fc9faa3-6d73-4861-b529-4bbdf03d219f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:52:02.186226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.186163    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1fc9faa3-6d73-4861-b529-4bbdf03d219f" (UID: "1fc9faa3-6d73-4861-b529-4bbdf03d219f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:52:02.186226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.186196    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-data" (OuterVolumeSpecName: "istio-data") pod "1fc9faa3-6d73-4861-b529-4bbdf03d219f" (UID: "1fc9faa3-6d73-4861-b529-4bbdf03d219f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:52:02.186493 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.186319    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-data\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:02.186493 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.186375    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:02.186493 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.186390    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-credential-socket\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:02.186493 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.186405    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-workload-socket\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:02.186764 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.186485    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1fc9faa3-6d73-4861-b529-4bbdf03d219f" (UID: "1fc9faa3-6d73-4861-b529-4bbdf03d219f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:52:02.188205 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.188176    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1fc9faa3-6d73-4861-b529-4bbdf03d219f" (UID: "1fc9faa3-6d73-4861-b529-4bbdf03d219f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:52:02.188329 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.188309    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1fc9faa3-6d73-4861-b529-4bbdf03d219f" (UID: "1fc9faa3-6d73-4861-b529-4bbdf03d219f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:52:02.188449 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.188429    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-kube-api-access-hvdwr" (OuterVolumeSpecName: "kube-api-access-hvdwr") pod "1fc9faa3-6d73-4861-b529-4bbdf03d219f" (UID: "1fc9faa3-6d73-4861-b529-4bbdf03d219f"). InnerVolumeSpecName "kube-api-access-hvdwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:52:02.188499 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.188484    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-token" (OuterVolumeSpecName: "istio-token") pod "1fc9faa3-6d73-4861-b529-4bbdf03d219f" (UID: "1fc9faa3-6d73-4861-b529-4bbdf03d219f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:52:02.257329 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.257296    2582 generic.go:356] "Generic (PLEG): container finished" podID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerID="30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef" exitCode=0
Sep 11 17:52:02.257506 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.257350    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"
Sep 11 17:52:02.257506 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.257385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" event={"ID":"1fc9faa3-6d73-4861-b529-4bbdf03d219f","Type":"ContainerDied","Data":"30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef"}
Sep 11 17:52:02.257506 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.257445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k" event={"ID":"1fc9faa3-6d73-4861-b529-4bbdf03d219f","Type":"ContainerDied","Data":"443251ea3085931102ae55d0c9f0f00ea5e9d7d4ac1948f44592b743a50c185a"}
Sep 11 17:52:02.257506 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.257474    2582 scope.go:117] "RemoveContainer" containerID="30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef"
Sep 11 17:52:02.267823 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.267789    2582 scope.go:117] "RemoveContainer" containerID="fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43"
Sep 11 17:52:02.277175 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.277141    2582 scope.go:117] "RemoveContainer" containerID="30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef"
Sep 11 17:52:02.277580 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:52:02.277556    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef\": container with ID starting with 30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef not found: ID does not exist" containerID="30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef"
Sep 11 17:52:02.277705 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.277587    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef"} err="failed to get container status \"30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef\": rpc error: code = NotFound desc = could not find container \"30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef\": container with ID starting with 30d1613f89381475a6db1373653fb31e91b94e68c8d22e3966ff9c4366ec05ef not found: ID does not exist"
Sep 11 17:52:02.277705 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.277605    2582 scope.go:117] "RemoveContainer" containerID="fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43"
Sep 11 17:52:02.277932 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:52:02.277897    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43\": container with ID starting with fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43 not found: ID does not exist" containerID="fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43"
Sep 11 17:52:02.277987 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.277938    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43"} err="failed to get container status \"fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43\": rpc error: code = NotFound desc = could not find container \"fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43\": container with ID starting with fc9b337ec046c9d8778b973a012ae39e198cb3b388892455a53d8784d652ba43 not found: ID does not exist"
Sep 11 17:52:02.287972 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.287929    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istiod-ca-cert\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:02.287972 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.287961    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-podinfo\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:02.287972 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.287972    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-token\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:02.287972 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.287987    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fc9faa3-6d73-4861-b529-4bbdf03d219f-istio-envoy\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:02.287972 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.287998    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvdwr\" (UniqueName: \"kubernetes.io/projected/1fc9faa3-6d73-4861-b529-4bbdf03d219f-kube-api-access-hvdwr\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:02.289947 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.289922    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"]
Sep 11 17:52:02.298591 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.298548    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-50ce3a65-openshift-default-5bb9cdbcdd-49f7k"]
Sep 11 17:52:02.465323 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.465289    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"]
Sep 11 17:52:02.465695 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.465682    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy"
Sep 11 17:52:02.465739 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.465696    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy"
Sep 11 17:52:02.465739 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.465708    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy"
Sep 11 17:52:02.465739 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.465713    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy"
Sep 11 17:52:02.465968 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.465771    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy"
Sep 11 17:52:02.465968 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.465779    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" containerName="istio-proxy"
Sep 11 17:52:02.470462 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.470435    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.474278 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.474255    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-3f5d8d77-serving-cert\""
Sep 11 17:52:02.482374 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.482290    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"]
Sep 11 17:52:02.489757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.489722    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mndf\" (UniqueName: \"kubernetes.io/projected/44a6cd04-c500-4805-b20a-267056485e66-kube-api-access-9mndf\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.489942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.489768    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-metrics-certs\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.489942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.489830    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-maas-api-tls\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.489942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.489886    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44a6cd04-c500-4805-b20a-267056485e66-openshift-service-ca\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.591389 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.591345    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44a6cd04-c500-4805-b20a-267056485e66-openshift-service-ca\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.591567 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.591471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mndf\" (UniqueName: \"kubernetes.io/projected/44a6cd04-c500-4805-b20a-267056485e66-kube-api-access-9mndf\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.591567 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.591495    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-metrics-certs\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.591567 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.591535    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-maas-api-tls\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.591730 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:52:02.591645    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-serving-cert: secret "maas-api-e2e-migrate-3f5d8d77-serving-cert" not found
Sep 11 17:52:02.591730 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:52:02.591713    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-maas-api-tls podName:44a6cd04-c500-4805-b20a-267056485e66 nodeName:}" failed. No retries permitted until 2026-09-11 17:52:03.091697395 +0000 UTC m=+790.979875719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-maas-api-tls") pod "maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4" (UID: "44a6cd04-c500-4805-b20a-267056485e66") : secret "maas-api-e2e-migrate-3f5d8d77-serving-cert" not found
Sep 11 17:52:02.592032 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.592011    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44a6cd04-c500-4805-b20a-267056485e66-openshift-service-ca\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.594263 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.594239    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-metrics-certs\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.604008 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.603974    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mndf\" (UniqueName: \"kubernetes.io/projected/44a6cd04-c500-4805-b20a-267056485e66-kube-api-access-9mndf\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:02.634880 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:02.634840    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fc9faa3-6d73-4861-b529-4bbdf03d219f" path="/var/lib/kubelet/pods/1fc9faa3-6d73-4861-b529-4bbdf03d219f/volumes"
Sep 11 17:52:03.098214 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.097337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-maas-api-tls\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:03.111769 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.111516    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-maas-api-tls\") pod \"maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:03.383072 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.382974    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:03.482407 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.482364    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"]
Sep 11 17:52:03.487203 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.487174    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:03.490193 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.490165    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-3f5d8d77\""
Sep 11 17:52:03.490364 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.490314    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-3f5d8d77-dockercfg-2dcr5\""
Sep 11 17:52:03.502067 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.501785    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd4d6\" (UniqueName: \"kubernetes.io/projected/19c82451-2f2c-4eef-a7a3-4fb01de0f520-kube-api-access-nd4d6\") pod \"payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc\" (UID: \"19c82451-2f2c-4eef-a7a3-4fb01de0f520\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:03.502067 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.501923    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19c82451-2f2c-4eef-a7a3-4fb01de0f520-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc\" (UID: \"19c82451-2f2c-4eef-a7a3-4fb01de0f520\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:03.503863 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.503838    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"]
Sep 11 17:52:03.603111 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.603040    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19c82451-2f2c-4eef-a7a3-4fb01de0f520-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc\" (UID: \"19c82451-2f2c-4eef-a7a3-4fb01de0f520\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:03.603267 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.603166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd4d6\" (UniqueName: \"kubernetes.io/projected/19c82451-2f2c-4eef-a7a3-4fb01de0f520-kube-api-access-nd4d6\") pod \"payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc\" (UID: \"19c82451-2f2c-4eef-a7a3-4fb01de0f520\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:03.603740 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.603720    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19c82451-2f2c-4eef-a7a3-4fb01de0f520-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc\" (UID: \"19c82451-2f2c-4eef-a7a3-4fb01de0f520\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:03.612406 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.612376    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd4d6\" (UniqueName: \"kubernetes.io/projected/19c82451-2f2c-4eef-a7a3-4fb01de0f520-kube-api-access-nd4d6\") pod \"payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc\" (UID: \"19c82451-2f2c-4eef-a7a3-4fb01de0f520\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:03.661897 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.661872    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"]
Sep 11 17:52:03.664009 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:52:03.663981    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a6cd04_c500_4805_b20a_267056485e66.slice/crio-4ccc6cc4cb3e7845fd3102e5e56fb1b57480fd46a043d882a3eb34d6ba45bf53 WatchSource:0}: Error finding container 4ccc6cc4cb3e7845fd3102e5e56fb1b57480fd46a043d882a3eb34d6ba45bf53: Status 404 returned error can't find the container with id 4ccc6cc4cb3e7845fd3102e5e56fb1b57480fd46a043d882a3eb34d6ba45bf53
Sep 11 17:52:03.803925 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:03.803886    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:04.079322 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:04.079235    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"]
Sep 11 17:52:04.082236 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:52:04.082199    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19c82451_2f2c_4eef_a7a3_4fb01de0f520.slice/crio-58819f8a5c756d3dff6d812b7978c85601720f69cfb6e191a58987a7fed3f047 WatchSource:0}: Error finding container 58819f8a5c756d3dff6d812b7978c85601720f69cfb6e191a58987a7fed3f047: Status 404 returned error can't find the container with id 58819f8a5c756d3dff6d812b7978c85601720f69cfb6e191a58987a7fed3f047
Sep 11 17:52:04.270603 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:04.270551    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc" event={"ID":"19c82451-2f2c-4eef-a7a3-4fb01de0f520","Type":"ContainerStarted","Data":"7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7"}
Sep 11 17:52:04.270808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:04.270622    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc" event={"ID":"19c82451-2f2c-4eef-a7a3-4fb01de0f520","Type":"ContainerStarted","Data":"58819f8a5c756d3dff6d812b7978c85601720f69cfb6e191a58987a7fed3f047"}
Sep 11 17:52:04.270808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:04.270674    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:04.272311 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:04.272281    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4" event={"ID":"44a6cd04-c500-4805-b20a-267056485e66","Type":"ContainerStarted","Data":"4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209"}
Sep 11 17:52:04.272458 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:04.272334    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4" event={"ID":"44a6cd04-c500-4805-b20a-267056485e66","Type":"ContainerStarted","Data":"4ccc6cc4cb3e7845fd3102e5e56fb1b57480fd46a043d882a3eb34d6ba45bf53"}
Sep 11 17:52:04.272458 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:04.272364    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:04.287459 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:04.287405    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc" podStartSLOduration=1.2873854759999999 podStartE2EDuration="1.287385476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:52:04.286528247 +0000 UTC m=+792.174706593" watchObservedRunningTime="2026-09-11 17:52:04.287385476 +0000 UTC m=+792.175563822"
Sep 11 17:52:04.305599 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:04.305537    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4" podStartSLOduration=1.855388924 podStartE2EDuration="2.305517334s" totalImagesPullingTime="450.12841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:52:04.304720564 +0000 UTC m=+792.192898910" watchObservedRunningTime="2026-09-11 17:52:04.305517334 +0000 UTC m=+792.193695713"
Sep 11 17:52:08.480768 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:08.480728    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:52:08.481315 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:08.480789    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:52:08.493741 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:08.493709    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:52:09.309490 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:09.309454    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 17:52:10.282980 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:10.282951    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:15.280149 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:15.280044    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:31.724509 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:31.724474    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"]
Sep 11 17:52:31.724957 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:31.724711    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4" podUID="44a6cd04-c500-4805-b20a-267056485e66" containerName="maas-api" containerID="cri-o://4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209" gracePeriod=30
Sep 11 17:52:31.793426 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:31.793390    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"]
Sep 11 17:52:31.793864 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:31.793753    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc" podUID="19c82451-2f2c-4eef-a7a3-4fb01de0f520" containerName="payload-pre-processing" containerID="cri-o://7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7" gracePeriod=30
Sep 11 17:52:32.051223 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.051197    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:32.069802 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.069778    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:32.176117 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.176050    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mndf\" (UniqueName: \"kubernetes.io/projected/44a6cd04-c500-4805-b20a-267056485e66-kube-api-access-9mndf\") pod \"44a6cd04-c500-4805-b20a-267056485e66\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") "
Sep 11 17:52:32.176319 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.176172    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19c82451-2f2c-4eef-a7a3-4fb01de0f520-plugins-config-volume\") pod \"19c82451-2f2c-4eef-a7a3-4fb01de0f520\" (UID: \"19c82451-2f2c-4eef-a7a3-4fb01de0f520\") "
Sep 11 17:52:32.176319 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.176222    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44a6cd04-c500-4805-b20a-267056485e66-openshift-service-ca\") pod \"44a6cd04-c500-4805-b20a-267056485e66\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") "
Sep 11 17:52:32.176450 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.176353    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd4d6\" (UniqueName: \"kubernetes.io/projected/19c82451-2f2c-4eef-a7a3-4fb01de0f520-kube-api-access-nd4d6\") pod \"19c82451-2f2c-4eef-a7a3-4fb01de0f520\" (UID: \"19c82451-2f2c-4eef-a7a3-4fb01de0f520\") "
Sep 11 17:52:32.176450 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.176398    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-metrics-certs\") pod \"44a6cd04-c500-4805-b20a-267056485e66\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") "
Sep 11 17:52:32.176450 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.176441    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-maas-api-tls\") pod \"44a6cd04-c500-4805-b20a-267056485e66\" (UID: \"44a6cd04-c500-4805-b20a-267056485e66\") "
Sep 11 17:52:32.176615 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.176581    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44a6cd04-c500-4805-b20a-267056485e66-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44a6cd04-c500-4805-b20a-267056485e66" (UID: "44a6cd04-c500-4805-b20a-267056485e66"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:52:32.176615 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.176590    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19c82451-2f2c-4eef-a7a3-4fb01de0f520-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "19c82451-2f2c-4eef-a7a3-4fb01de0f520" (UID: "19c82451-2f2c-4eef-a7a3-4fb01de0f520"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:52:32.176797 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.176780    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19c82451-2f2c-4eef-a7a3-4fb01de0f520-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:32.176853 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.176803    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44a6cd04-c500-4805-b20a-267056485e66-openshift-service-ca\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:32.178563 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.178534    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a6cd04-c500-4805-b20a-267056485e66-kube-api-access-9mndf" (OuterVolumeSpecName: "kube-api-access-9mndf") pod "44a6cd04-c500-4805-b20a-267056485e66" (UID: "44a6cd04-c500-4805-b20a-267056485e66"). InnerVolumeSpecName "kube-api-access-9mndf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:52:32.178685 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.178631    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "44a6cd04-c500-4805-b20a-267056485e66" (UID: "44a6cd04-c500-4805-b20a-267056485e66"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:52:32.179041 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.179021    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44a6cd04-c500-4805-b20a-267056485e66" (UID: "44a6cd04-c500-4805-b20a-267056485e66"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:52:32.179134 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.179039    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19c82451-2f2c-4eef-a7a3-4fb01de0f520-kube-api-access-nd4d6" (OuterVolumeSpecName: "kube-api-access-nd4d6") pod "19c82451-2f2c-4eef-a7a3-4fb01de0f520" (UID: "19c82451-2f2c-4eef-a7a3-4fb01de0f520"). InnerVolumeSpecName "kube-api-access-nd4d6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:52:32.277573 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.277477    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd4d6\" (UniqueName: \"kubernetes.io/projected/19c82451-2f2c-4eef-a7a3-4fb01de0f520-kube-api-access-nd4d6\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:32.277573 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.277510    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-metrics-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:32.277573 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.277523    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44a6cd04-c500-4805-b20a-267056485e66-maas-api-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:32.277573 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.277534    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mndf\" (UniqueName: \"kubernetes.io/projected/44a6cd04-c500-4805-b20a-267056485e66-kube-api-access-9mndf\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:52:32.435486 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.435451    2582 generic.go:356] "Generic (PLEG): container finished" podID="19c82451-2f2c-4eef-a7a3-4fb01de0f520" containerID="7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7" exitCode=0
Sep 11 17:52:32.435678 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.435498    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"
Sep 11 17:52:32.435678 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.435533    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc" event={"ID":"19c82451-2f2c-4eef-a7a3-4fb01de0f520","Type":"ContainerDied","Data":"7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7"}
Sep 11 17:52:32.435678 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.435604    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc" event={"ID":"19c82451-2f2c-4eef-a7a3-4fb01de0f520","Type":"ContainerDied","Data":"58819f8a5c756d3dff6d812b7978c85601720f69cfb6e191a58987a7fed3f047"}
Sep 11 17:52:32.435678 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.435633    2582 scope.go:117] "RemoveContainer" containerID="7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7"
Sep 11 17:52:32.437100 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.437062    2582 generic.go:356] "Generic (PLEG): container finished" podID="44a6cd04-c500-4805-b20a-267056485e66" containerID="4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209" exitCode=0
Sep 11 17:52:32.437235 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.437114    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4" event={"ID":"44a6cd04-c500-4805-b20a-267056485e66","Type":"ContainerDied","Data":"4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209"}
Sep 11 17:52:32.437235 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.437132    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"
Sep 11 17:52:32.437235 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.437175    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4" event={"ID":"44a6cd04-c500-4805-b20a-267056485e66","Type":"ContainerDied","Data":"4ccc6cc4cb3e7845fd3102e5e56fb1b57480fd46a043d882a3eb34d6ba45bf53"}
Sep 11 17:52:32.445387 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.445366    2582 scope.go:117] "RemoveContainer" containerID="7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7"
Sep 11 17:52:32.445665 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:52:32.445642    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7\": container with ID starting with 7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7 not found: ID does not exist" containerID="7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7"
Sep 11 17:52:32.445760 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.445676    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7"} err="failed to get container status \"7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7\": rpc error: code = NotFound desc = could not find container \"7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7\": container with ID starting with 7f1fb0f0fb0580232de89f1edc9be12309fa9897ee5550132cd2b3a1c12a6cf7 not found: ID does not exist"
Sep 11 17:52:32.445760 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.445701    2582 scope.go:117] "RemoveContainer" containerID="4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209"
Sep 11 17:52:32.453952 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.453936    2582 scope.go:117] "RemoveContainer" containerID="4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209"
Sep 11 17:52:32.454269 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:52:32.454249    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209\": container with ID starting with 4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209 not found: ID does not exist" containerID="4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209"
Sep 11 17:52:32.454334 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.454277    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209"} err="failed to get container status \"4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209\": rpc error: code = NotFound desc = could not find container \"4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209\": container with ID starting with 4e568a3eca36b04584994a9eea5192bf3ff8b00185a7e2ff119ccb35451ad209 not found: ID does not exist"
Sep 11 17:52:32.459939 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.459915    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"]
Sep 11 17:52:32.468223 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.468202    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3f5d8d77-5d5bb6478-zrgcc"]
Sep 11 17:52:32.482667 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.482629    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"]
Sep 11 17:52:32.484422 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.484397    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3f5d8d77-799665b7d5-drdj4"]
Sep 11 17:52:32.633541 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.633465    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19c82451-2f2c-4eef-a7a3-4fb01de0f520" path="/var/lib/kubelet/pods/19c82451-2f2c-4eef-a7a3-4fb01de0f520/volumes"
Sep 11 17:52:32.633868 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:32.633854    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44a6cd04-c500-4805-b20a-267056485e66" path="/var/lib/kubelet/pods/44a6cd04-c500-4805-b20a-267056485e66/volumes"
Sep 11 17:52:37.665484 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.665442    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"]
Sep 11 17:52:37.665955 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.665841    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19c82451-2f2c-4eef-a7a3-4fb01de0f520" containerName="payload-pre-processing"
Sep 11 17:52:37.665955 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.665859    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="19c82451-2f2c-4eef-a7a3-4fb01de0f520" containerName="payload-pre-processing"
Sep 11 17:52:37.665955 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.665888    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44a6cd04-c500-4805-b20a-267056485e66" containerName="maas-api"
Sep 11 17:52:37.665955 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.665896    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="44a6cd04-c500-4805-b20a-267056485e66" containerName="maas-api"
Sep 11 17:52:37.666147 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.665971    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="44a6cd04-c500-4805-b20a-267056485e66" containerName="maas-api"
Sep 11 17:52:37.666147 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.665982    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="19c82451-2f2c-4eef-a7a3-4fb01de0f520" containerName="payload-pre-processing"
Sep 11 17:52:37.670542 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.670524    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.673297 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.673274    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-feb821e2-serving-cert\""
Sep 11 17:52:37.683237 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.683193    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"]
Sep 11 17:52:37.735671 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.735636    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kqbv\" (UniqueName: \"kubernetes.io/projected/7a916c89-80b5-4ebc-98b2-8b1962932aa2-kube-api-access-6kqbv\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.735879 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.735679    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-metrics-certs\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.735879 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.735811    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-maas-api-tls\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.735879 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.735855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a916c89-80b5-4ebc-98b2-8b1962932aa2-openshift-service-ca\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.837105 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.837047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a916c89-80b5-4ebc-98b2-8b1962932aa2-openshift-service-ca\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.837311 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.837118    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kqbv\" (UniqueName: \"kubernetes.io/projected/7a916c89-80b5-4ebc-98b2-8b1962932aa2-kube-api-access-6kqbv\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.837311 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.837137    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-metrics-certs\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.837311 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.837205    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-maas-api-tls\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.837311 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:52:37.837303    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-serving-cert: secret "maas-api-e2e-ait-feb821e2-serving-cert" not found
Sep 11 17:52:37.837538 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:52:37.837366    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-maas-api-tls podName:7a916c89-80b5-4ebc-98b2-8b1962932aa2 nodeName:}" failed. No retries permitted until 2026-09-11 17:52:38.337349579 +0000 UTC m=+826.225527925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-maas-api-tls") pod "maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg" (UID: "7a916c89-80b5-4ebc-98b2-8b1962932aa2") : secret "maas-api-e2e-ait-feb821e2-serving-cert" not found
Sep 11 17:52:37.837725 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.837703    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a916c89-80b5-4ebc-98b2-8b1962932aa2-openshift-service-ca\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.839788 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.839769    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-metrics-certs\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:37.849003 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:37.848978    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kqbv\" (UniqueName: \"kubernetes.io/projected/7a916c89-80b5-4ebc-98b2-8b1962932aa2-kube-api-access-6kqbv\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:38.327167 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.327130    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"]
Sep 11 17:52:38.330941 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.330915    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:52:38.334536 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.334508    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-feb821e2\""
Sep 11 17:52:38.334692 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.334542    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-feb821e2-dockercfg-ksclk\""
Sep 11 17:52:38.341758 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.341723    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-maas-api-tls\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:38.341913 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.341793    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnfrk\" (UniqueName: \"kubernetes.io/projected/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-kube-api-access-dnfrk\") pod \"payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w\" (UID: \"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:52:38.341913 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.341866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w\" (UID: \"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:52:38.342767 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.342741    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"]
Sep 11 17:52:38.344525 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.344500    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-maas-api-tls\") pod \"maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:38.442564 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.442526    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnfrk\" (UniqueName: \"kubernetes.io/projected/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-kube-api-access-dnfrk\") pod \"payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w\" (UID: \"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:52:38.442767 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.442598    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w\" (UID: \"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:52:38.443378 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.443356    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w\" (UID: \"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:52:38.452397 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.452364    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnfrk\" (UniqueName: \"kubernetes.io/projected/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-kube-api-access-dnfrk\") pod \"payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w\" (UID: \"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:52:38.581154 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.581035    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:38.642586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.642552    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:52:38.862387 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.862359    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"]
Sep 11 17:52:38.864896 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:52:38.864868    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a916c89_80b5_4ebc_98b2_8b1962932aa2.slice/crio-dc8aefb67274e1b20883520e5493f32901bbc31003fcc4de01a70fa8f2957a7d WatchSource:0}: Error finding container dc8aefb67274e1b20883520e5493f32901bbc31003fcc4de01a70fa8f2957a7d: Status 404 returned error can't find the container with id dc8aefb67274e1b20883520e5493f32901bbc31003fcc4de01a70fa8f2957a7d
Sep 11 17:52:38.921378 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:38.921352    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"]
Sep 11 17:52:38.923326 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:52:38.923252    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3e460a_0e6f_4bee_b0b4_dcb5c54df15c.slice/crio-e07c76ec5d90db1602233a7738b691670e33d9c024d4c80cc10438615d11795e WatchSource:0}: Error finding container e07c76ec5d90db1602233a7738b691670e33d9c024d4c80cc10438615d11795e: Status 404 returned error can't find the container with id e07c76ec5d90db1602233a7738b691670e33d9c024d4c80cc10438615d11795e
Sep 11 17:52:39.476606 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:39.476569    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w" event={"ID":"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c","Type":"ContainerStarted","Data":"6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c"}
Sep 11 17:52:39.476810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:39.476622    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w" event={"ID":"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c","Type":"ContainerStarted","Data":"e07c76ec5d90db1602233a7738b691670e33d9c024d4c80cc10438615d11795e"}
Sep 11 17:52:39.476810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:39.476644    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:52:39.478196 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:39.478170    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg" event={"ID":"7a916c89-80b5-4ebc-98b2-8b1962932aa2","Type":"ContainerStarted","Data":"09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08"}
Sep 11 17:52:39.478298 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:39.478213    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg" event={"ID":"7a916c89-80b5-4ebc-98b2-8b1962932aa2","Type":"ContainerStarted","Data":"dc8aefb67274e1b20883520e5493f32901bbc31003fcc4de01a70fa8f2957a7d"}
Sep 11 17:52:39.478298 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:39.478232    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:39.494267 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:39.494218    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w" podStartSLOduration=1.494201215 podStartE2EDuration="1.494201215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:52:39.492712427 +0000 UTC m=+827.380890774" watchObservedRunningTime="2026-09-11 17:52:39.494201215 +0000 UTC m=+827.382379561"
Sep 11 17:52:39.512744 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:39.512695    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg" podStartSLOduration=2.056404096 podStartE2EDuration="2.512677683s" totalImagesPullingTime="456.273587ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:52:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:52:39.510320316 +0000 UTC m=+827.398498661" watchObservedRunningTime="2026-09-11 17:52:39.512677683 +0000 UTC m=+827.400856029"
Sep 11 17:52:45.489035 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:45.489004    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:52:50.486458 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:52:50.486428    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:53:07.768647 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:07.768601    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"]
Sep 11 17:53:07.769635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:07.768881    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg" podUID="7a916c89-80b5-4ebc-98b2-8b1962932aa2" containerName="maas-api" containerID="cri-o://09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08" gracePeriod=30
Sep 11 17:53:07.844448 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:07.844412    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"]
Sep 11 17:53:07.844724 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:07.844662    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w" podUID="9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c" containerName="payload-pre-processing" containerID="cri-o://6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c" gracePeriod=30
Sep 11 17:53:08.092045 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.092020    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:53:08.128686 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.128653    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:53:08.176423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.176384    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"]
Sep 11 17:53:08.176777 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.176765    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a916c89-80b5-4ebc-98b2-8b1962932aa2" containerName="maas-api"
Sep 11 17:53:08.176836 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.176779    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a916c89-80b5-4ebc-98b2-8b1962932aa2" containerName="maas-api"
Sep 11 17:53:08.176836 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.176794    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c" containerName="payload-pre-processing"
Sep 11 17:53:08.176836 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.176800    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c" containerName="payload-pre-processing"
Sep 11 17:53:08.176933 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.176862    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c" containerName="payload-pre-processing"
Sep 11 17:53:08.176933 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.176869    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a916c89-80b5-4ebc-98b2-8b1962932aa2" containerName="maas-api"
Sep 11 17:53:08.179810 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.179791    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.182839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.182809    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-37899e\"/\"default-dockercfg-7c8lg\""
Sep 11 17:53:08.182839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.182831    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-37899e\"/\"autores-model-37899e-kserve-self-signed-certs\""
Sep 11 17:53:08.183039 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.182817    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-37899e\"/\"openshift-service-ca.crt\""
Sep 11 17:53:08.183039 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.182822    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-37899e\"/\"kube-root-ca.crt\""
Sep 11 17:53:08.191005 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.190978    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"]
Sep 11 17:53:08.224954 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.224916    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-metrics-certs\") pod \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") "
Sep 11 17:53:08.225170 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.224978    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kqbv\" (UniqueName: \"kubernetes.io/projected/7a916c89-80b5-4ebc-98b2-8b1962932aa2-kube-api-access-6kqbv\") pod \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") "
Sep 11 17:53:08.225170 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.225026    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a916c89-80b5-4ebc-98b2-8b1962932aa2-openshift-service-ca\") pod \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") "
Sep 11 17:53:08.225170 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.225061    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-maas-api-tls\") pod \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\" (UID: \"7a916c89-80b5-4ebc-98b2-8b1962932aa2\") "
Sep 11 17:53:08.225170 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.225117    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnfrk\" (UniqueName: \"kubernetes.io/projected/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-kube-api-access-dnfrk\") pod \"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c\" (UID: \"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c\") "
Sep 11 17:53:08.225170 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.225146    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-plugins-config-volume\") pod \"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c\" (UID: \"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c\") "
Sep 11 17:53:08.225436 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.225406    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a916c89-80b5-4ebc-98b2-8b1962932aa2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7a916c89-80b5-4ebc-98b2-8b1962932aa2" (UID: "7a916c89-80b5-4ebc-98b2-8b1962932aa2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:53:08.225551 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.225532    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a916c89-80b5-4ebc-98b2-8b1962932aa2-openshift-service-ca\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:08.225655 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.225626    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c" (UID: "9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:53:08.227415 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.227382    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7a916c89-80b5-4ebc-98b2-8b1962932aa2" (UID: "7a916c89-80b5-4ebc-98b2-8b1962932aa2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:53:08.227415 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.227400    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7a916c89-80b5-4ebc-98b2-8b1962932aa2" (UID: "7a916c89-80b5-4ebc-98b2-8b1962932aa2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:53:08.227737 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.227721    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a916c89-80b5-4ebc-98b2-8b1962932aa2-kube-api-access-6kqbv" (OuterVolumeSpecName: "kube-api-access-6kqbv") pod "7a916c89-80b5-4ebc-98b2-8b1962932aa2" (UID: "7a916c89-80b5-4ebc-98b2-8b1962932aa2"). InnerVolumeSpecName "kube-api-access-6kqbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:53:08.227799 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.227733    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-kube-api-access-dnfrk" (OuterVolumeSpecName: "kube-api-access-dnfrk") pod "9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c" (UID: "9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c"). InnerVolumeSpecName "kube-api-access-dnfrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:53:08.326681 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326591    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/162fbab0-8e47-4c0e-8abe-23806ea01487-tls-certs\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.326681 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326633    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-home\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.326681 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-tmp-dir\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.326681 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326672    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-model-cache\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.326942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326748    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-dshm\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.326942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd75t\" (UniqueName: \"kubernetes.io/projected/162fbab0-8e47-4c0e-8abe-23806ea01487-kube-api-access-pd75t\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.326942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326898    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-maas-api-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:08.326942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326916    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnfrk\" (UniqueName: \"kubernetes.io/projected/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-kube-api-access-dnfrk\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:08.326942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326926    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:08.326942 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326936    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a916c89-80b5-4ebc-98b2-8b1962932aa2-metrics-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:08.327169 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.326946    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kqbv\" (UniqueName: \"kubernetes.io/projected/7a916c89-80b5-4ebc-98b2-8b1962932aa2-kube-api-access-6kqbv\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:08.428539 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.428499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/162fbab0-8e47-4c0e-8abe-23806ea01487-tls-certs\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.428718 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.428552    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-home\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.428718 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.428583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-tmp-dir\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.428718 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.428613    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-model-cache\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.428885 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.428785    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-dshm\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.428885 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.428819    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd75t\" (UniqueName: \"kubernetes.io/projected/162fbab0-8e47-4c0e-8abe-23806ea01487-kube-api-access-pd75t\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.429549 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.429518    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-home\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.429805 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.429762    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-tmp-dir\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.429932 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.429906    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-model-cache\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.431589 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.431566    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-dshm\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.431704 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.431689    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/162fbab0-8e47-4c0e-8abe-23806ea01487-tls-certs\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.440236 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.440208    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd75t\" (UniqueName: \"kubernetes.io/projected/162fbab0-8e47-4c0e-8abe-23806ea01487-kube-api-access-pd75t\") pod \"autores-model-37899e-kserve-544dd4547-m2kzm\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") " pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.490404 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.490363    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:08.637659 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.637570    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a916c89-80b5-4ebc-98b2-8b1962932aa2" containerID="09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08" exitCode=0
Sep 11 17:53:08.637659 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.637634    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"
Sep 11 17:53:08.637901 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.637661    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg" event={"ID":"7a916c89-80b5-4ebc-98b2-8b1962932aa2","Type":"ContainerDied","Data":"09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08"}
Sep 11 17:53:08.637901 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.637717    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg" event={"ID":"7a916c89-80b5-4ebc-98b2-8b1962932aa2","Type":"ContainerDied","Data":"dc8aefb67274e1b20883520e5493f32901bbc31003fcc4de01a70fa8f2957a7d"}
Sep 11 17:53:08.637901 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.637746    2582 scope.go:117] "RemoveContainer" containerID="09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08"
Sep 11 17:53:08.639234 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.639209    2582 generic.go:356] "Generic (PLEG): container finished" podID="9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c" containerID="6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c" exitCode=0
Sep 11 17:53:08.639345 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.639254    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w" event={"ID":"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c","Type":"ContainerDied","Data":"6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c"}
Sep 11 17:53:08.639345 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.639268    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"
Sep 11 17:53:08.639345 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.639298    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w" event={"ID":"9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c","Type":"ContainerDied","Data":"e07c76ec5d90db1602233a7738b691670e33d9c024d4c80cc10438615d11795e"}
Sep 11 17:53:08.647866 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.647845    2582 scope.go:117] "RemoveContainer" containerID="09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08"
Sep 11 17:53:08.648222 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:53:08.648199    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08\": container with ID starting with 09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08 not found: ID does not exist" containerID="09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08"
Sep 11 17:53:08.648319 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.648228    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08"} err="failed to get container status \"09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08\": rpc error: code = NotFound desc = could not find container \"09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08\": container with ID starting with 09c2996e40876f9ff42df429abde8db9ea935a8e45687ea0a884209b37c88a08 not found: ID does not exist"
Sep 11 17:53:08.648319 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.648246    2582 scope.go:117] "RemoveContainer" containerID="6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c"
Sep 11 17:53:08.657070 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.657050    2582 scope.go:117] "RemoveContainer" containerID="6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c"
Sep 11 17:53:08.657440 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:53:08.657421    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c\": container with ID starting with 6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c not found: ID does not exist" containerID="6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c"
Sep 11 17:53:08.657490 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.657448    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c"} err="failed to get container status \"6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c\": rpc error: code = NotFound desc = could not find container \"6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c\": container with ID starting with 6bf1ecb81a635ac840bd4de48fc3ce793c94eeedc2c39e54ef78e7ec49e6975c not found: ID does not exist"
Sep 11 17:53:08.668594 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.668560    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"]
Sep 11 17:53:08.675897 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.675866    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-feb821e2-69c9c8b5bb-gqk5w"]
Sep 11 17:53:08.691177 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.691145    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"]
Sep 11 17:53:08.702183 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.702149    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-feb821e2-65f86f46bf-b54bg"]
Sep 11 17:53:08.768107 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:08.768063    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"]
Sep 11 17:53:08.770334 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:53:08.770293    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod162fbab0_8e47_4c0e_8abe_23806ea01487.slice/crio-3c93de1bf9f026423317047463e42dd50a2ad32aad3ddbd8e6f10ef23cc84a43 WatchSource:0}: Error finding container 3c93de1bf9f026423317047463e42dd50a2ad32aad3ddbd8e6f10ef23cc84a43: Status 404 returned error can't find the container with id 3c93de1bf9f026423317047463e42dd50a2ad32aad3ddbd8e6f10ef23cc84a43
Sep 11 17:53:09.646863 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:09.646826    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm" event={"ID":"162fbab0-8e47-4c0e-8abe-23806ea01487","Type":"ContainerStarted","Data":"92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016"}
Sep 11 17:53:09.646863 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:09.646869    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm" event={"ID":"162fbab0-8e47-4c0e-8abe-23806ea01487","Type":"ContainerStarted","Data":"3c93de1bf9f026423317047463e42dd50a2ad32aad3ddbd8e6f10ef23cc84a43"}
Sep 11 17:53:09.667348 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:09.667292    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm" podStartSLOduration=1.667269602 podStartE2EDuration="1.667269602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:53:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:53:09.665993338 +0000 UTC m=+857.554171681" watchObservedRunningTime="2026-09-11 17:53:09.667269602 +0000 UTC m=+857.555447959"
Sep 11 17:53:10.635331 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:10.635297    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a916c89-80b5-4ebc-98b2-8b1962932aa2" path="/var/lib/kubelet/pods/7a916c89-80b5-4ebc-98b2-8b1962932aa2/volumes"
Sep 11 17:53:10.635703 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:10.635677    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c" path="/var/lib/kubelet/pods/9c3e460a-0e6f-4bee-b0b4-dcb5c54df15c/volumes"
Sep 11 17:53:18.491006 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.490958    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:18.491006 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.491017    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:18.504690 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.504659    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:18.511068 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.511034    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"]
Sep 11 17:53:18.514350 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.514331    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:18.517409 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.517340    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-5f786284-dockercfg-fm6rl\""
Sep 11 17:53:18.517958 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.517934    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-5f786284\""
Sep 11 17:53:18.524011 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.523983    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-plugins-config-volume\") pod \"payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29\" (UID: \"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461\") " pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:18.524140 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.524024    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9766d\" (UniqueName: \"kubernetes.io/projected/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-kube-api-access-9766d\") pod \"payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29\" (UID: \"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461\") " pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:18.528217 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.528191    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"]
Sep 11 17:53:18.624707 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.624672    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-plugins-config-volume\") pod \"payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29\" (UID: \"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461\") " pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:18.624707 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.624708    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9766d\" (UniqueName: \"kubernetes.io/projected/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-kube-api-access-9766d\") pod \"payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29\" (UID: \"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461\") " pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:18.625422 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.625401    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-plugins-config-volume\") pod \"payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29\" (UID: \"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461\") " pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:18.634518 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.634481    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9766d\" (UniqueName: \"kubernetes.io/projected/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-kube-api-access-9766d\") pod \"payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29\" (UID: \"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461\") " pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:18.701556 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.701520    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:53:18.825671 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:18.825587    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:19.103032 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:19.102936    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"]
Sep 11 17:53:19.106553 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:53:19.106521    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4d7655_e2fc_4e5f_8675_0eb67b1d7461.slice/crio-47114ff4b39abdd940f5adee455addb310da1c79d88a02869099c31277b03146 WatchSource:0}: Error finding container 47114ff4b39abdd940f5adee455addb310da1c79d88a02869099c31277b03146: Status 404 returned error can't find the container with id 47114ff4b39abdd940f5adee455addb310da1c79d88a02869099c31277b03146
Sep 11 17:53:19.698635 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:19.698593    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29" event={"ID":"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461","Type":"ContainerStarted","Data":"218e8e756088b331f96f55148307b6c39c22da5f74b0ef059860ea4fcde4563c"}
Sep 11 17:53:19.699135 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:19.698665    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29" event={"ID":"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461","Type":"ContainerStarted","Data":"47114ff4b39abdd940f5adee455addb310da1c79d88a02869099c31277b03146"}
Sep 11 17:53:19.719637 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:19.719579    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29" podStartSLOduration=1.719559286 podStartE2EDuration="1.719559286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:53:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:53:19.717518081 +0000 UTC m=+867.605696505" watchObservedRunningTime="2026-09-11 17:53:19.719559286 +0000 UTC m=+867.607737631"
Sep 11 17:53:20.703531 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:20.703496    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:31.711651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:31.711622    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:45.877585 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.877485    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"]
Sep 11 17:53:45.881985 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.881958    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:45.886697 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.886662    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-715253\"/\"openshift-service-ca.crt\""
Sep 11 17:53:45.886875 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.886695    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-715253\"/\"test-model-715253-kserve-self-signed-certs\""
Sep 11 17:53:45.886875 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.886715    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-715253\"/\"default-dockercfg-ghdb4\""
Sep 11 17:53:45.886875 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.886671    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-715253\"/\"kube-root-ca.crt\""
Sep 11 17:53:45.892144 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.892116    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"]
Sep 11 17:53:45.992936 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.992904    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-dshm\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:45.993119 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.992978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-model-cache\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:45.993119 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.993007    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44m2g\" (UniqueName: \"kubernetes.io/projected/63efa942-3daf-42cf-8667-cb2f88d4c5da-kube-api-access-44m2g\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:45.993119 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.993024    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63efa942-3daf-42cf-8667-cb2f88d4c5da-tls-certs\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:45.993269 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.993173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-tmp-dir\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:45.993269 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:45.993204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-home\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.093694 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.093654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-model-cache\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.093694 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.093698    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44m2g\" (UniqueName: \"kubernetes.io/projected/63efa942-3daf-42cf-8667-cb2f88d4c5da-kube-api-access-44m2g\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.093923 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.093724    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63efa942-3daf-42cf-8667-cb2f88d4c5da-tls-certs\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.093923 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.093817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-tmp-dir\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.093923 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.093844    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-home\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.093923 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.093869    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-dshm\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.094217 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.094194    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-model-cache\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.094279 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.094223    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-home\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.094279 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.094247    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-tmp-dir\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.096283 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.096254    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-dshm\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.096528 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.096514    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63efa942-3daf-42cf-8667-cb2f88d4c5da-tls-certs\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.105459 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.105430    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44m2g\" (UniqueName: \"kubernetes.io/projected/63efa942-3daf-42cf-8667-cb2f88d4c5da-kube-api-access-44m2g\") pod \"test-model-715253-kserve-6d854f5c79-6gt7h\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") " pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.212169 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.212127    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:46.489366 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.489339    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"]
Sep 11 17:53:46.491387 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:53:46.491359    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63efa942_3daf_42cf_8667_cb2f88d4c5da.slice/crio-81d2625e16c1df0330214c01c73d4d3b6fbc8eda7fcdd1657959ae5ce7316be4 WatchSource:0}: Error finding container 81d2625e16c1df0330214c01c73d4d3b6fbc8eda7fcdd1657959ae5ce7316be4: Status 404 returned error can't find the container with id 81d2625e16c1df0330214c01c73d4d3b6fbc8eda7fcdd1657959ae5ce7316be4
Sep 11 17:53:46.842061 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.841956    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h" event={"ID":"63efa942-3daf-42cf-8667-cb2f88d4c5da","Type":"ContainerStarted","Data":"435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040"}
Sep 11 17:53:46.842061 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.842002    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h" event={"ID":"63efa942-3daf-42cf-8667-cb2f88d4c5da","Type":"ContainerStarted","Data":"81d2625e16c1df0330214c01c73d4d3b6fbc8eda7fcdd1657959ae5ce7316be4"}
Sep 11 17:53:46.863355 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:46.863302    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h" podStartSLOduration=1.863287708 podStartE2EDuration="1.863287708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:53:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:53:46.862327311 +0000 UTC m=+894.750505657" watchObservedRunningTime="2026-09-11 17:53:46.863287708 +0000 UTC m=+894.751466054"
Sep 11 17:53:54.688380 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:54.688332    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"]
Sep 11 17:53:54.688894 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:54.688606    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29" podUID="bf4d7655-e2fc-4e5f-8675-0eb67b1d7461" containerName="payload-processing" containerID="cri-o://218e8e756088b331f96f55148307b6c39c22da5f74b0ef059860ea4fcde4563c" gracePeriod=30
Sep 11 17:53:54.795771 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:54.795724    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"]
Sep 11 17:53:54.796878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:54.795968    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph" podUID="68c857a9-3cab-47a1-a15a-be0482ac0c87" containerName="maas-api" containerID="cri-o://ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351" gracePeriod=30
Sep 11 17:53:54.862853 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:54.862660    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"]
Sep 11 17:53:54.864058 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:54.863990    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b" podUID="42814990-b4d0-47a7-a687-ef0a597e7f3d" containerName="payload-processing" containerID="cri-o://79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448" gracePeriod=30
Sep 11 17:53:54.886603 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:54.886473    2582 generic.go:356] "Generic (PLEG): container finished" podID="bf4d7655-e2fc-4e5f-8675-0eb67b1d7461" containerID="218e8e756088b331f96f55148307b6c39c22da5f74b0ef059860ea4fcde4563c" exitCode=0
Sep 11 17:53:54.886603 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:54.886527    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29" event={"ID":"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461","Type":"ContainerDied","Data":"218e8e756088b331f96f55148307b6c39c22da5f74b0ef059860ea4fcde4563c"}
Sep 11 17:53:54.987581 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:54.987556    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:55.085851 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.085604    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-plugins-config-volume\") pod \"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461\" (UID: \"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461\") "
Sep 11 17:53:55.085851 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.085701    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9766d\" (UniqueName: \"kubernetes.io/projected/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-kube-api-access-9766d\") pod \"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461\" (UID: \"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461\") "
Sep 11 17:53:55.086125 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.086038    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf4d7655-e2fc-4e5f-8675-0eb67b1d7461" (UID: "bf4d7655-e2fc-4e5f-8675-0eb67b1d7461"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:53:55.086423 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.086393    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.089314 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.089286    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-kube-api-access-9766d" (OuterVolumeSpecName: "kube-api-access-9766d") pod "bf4d7655-e2fc-4e5f-8675-0eb67b1d7461" (UID: "bf4d7655-e2fc-4e5f-8675-0eb67b1d7461"). InnerVolumeSpecName "kube-api-access-9766d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:53:55.133176 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.133151    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:53:55.154841 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.154806    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:53:55.187807 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.187772    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9766d\" (UniqueName: \"kubernetes.io/projected/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461-kube-api-access-9766d\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.288448 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.288356    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42814990-b4d0-47a7-a687-ef0a597e7f3d-plugins-config-volume\") pod \"42814990-b4d0-47a7-a687-ef0a597e7f3d\" (UID: \"42814990-b4d0-47a7-a687-ef0a597e7f3d\") "
Sep 11 17:53:55.288448 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.288415    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-metrics-certs\") pod \"68c857a9-3cab-47a1-a15a-be0482ac0c87\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") "
Sep 11 17:53:55.288448 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.288446    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-maas-api-tls\") pod \"68c857a9-3cab-47a1-a15a-be0482ac0c87\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") "
Sep 11 17:53:55.288813 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.288572    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68c857a9-3cab-47a1-a15a-be0482ac0c87-openshift-service-ca\") pod \"68c857a9-3cab-47a1-a15a-be0482ac0c87\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") "
Sep 11 17:53:55.288813 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.288619    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzwz2\" (UniqueName: \"kubernetes.io/projected/42814990-b4d0-47a7-a687-ef0a597e7f3d-kube-api-access-vzwz2\") pod \"42814990-b4d0-47a7-a687-ef0a597e7f3d\" (UID: \"42814990-b4d0-47a7-a687-ef0a597e7f3d\") "
Sep 11 17:53:55.288813 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.288701    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fqv4\" (UniqueName: \"kubernetes.io/projected/68c857a9-3cab-47a1-a15a-be0482ac0c87-kube-api-access-9fqv4\") pod \"68c857a9-3cab-47a1-a15a-be0482ac0c87\" (UID: \"68c857a9-3cab-47a1-a15a-be0482ac0c87\") "
Sep 11 17:53:55.288975 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.288839    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42814990-b4d0-47a7-a687-ef0a597e7f3d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "42814990-b4d0-47a7-a687-ef0a597e7f3d" (UID: "42814990-b4d0-47a7-a687-ef0a597e7f3d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:53:55.288975 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.288916    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68c857a9-3cab-47a1-a15a-be0482ac0c87-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68c857a9-3cab-47a1-a15a-be0482ac0c87" (UID: "68c857a9-3cab-47a1-a15a-be0482ac0c87"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:53:55.289165 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.289140    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42814990-b4d0-47a7-a687-ef0a597e7f3d-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.289165 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.289164    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68c857a9-3cab-47a1-a15a-be0482ac0c87-openshift-service-ca\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.290761 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.290731    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "68c857a9-3cab-47a1-a15a-be0482ac0c87" (UID: "68c857a9-3cab-47a1-a15a-be0482ac0c87"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:53:55.291239 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.291201    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68c857a9-3cab-47a1-a15a-be0482ac0c87-kube-api-access-9fqv4" (OuterVolumeSpecName: "kube-api-access-9fqv4") pod "68c857a9-3cab-47a1-a15a-be0482ac0c87" (UID: "68c857a9-3cab-47a1-a15a-be0482ac0c87"). InnerVolumeSpecName "kube-api-access-9fqv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:53:55.291366 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.291348    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68c857a9-3cab-47a1-a15a-be0482ac0c87" (UID: "68c857a9-3cab-47a1-a15a-be0482ac0c87"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:53:55.291366 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.291348    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42814990-b4d0-47a7-a687-ef0a597e7f3d-kube-api-access-vzwz2" (OuterVolumeSpecName: "kube-api-access-vzwz2") pod "42814990-b4d0-47a7-a687-ef0a597e7f3d" (UID: "42814990-b4d0-47a7-a687-ef0a597e7f3d"). InnerVolumeSpecName "kube-api-access-vzwz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:53:55.390594 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.390555    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fqv4\" (UniqueName: \"kubernetes.io/projected/68c857a9-3cab-47a1-a15a-be0482ac0c87-kube-api-access-9fqv4\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.390594 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.390586    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-metrics-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.390594 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.390595    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c857a9-3cab-47a1-a15a-be0482ac0c87-maas-api-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.390594 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.390603    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzwz2\" (UniqueName: \"kubernetes.io/projected/42814990-b4d0-47a7-a687-ef0a597e7f3d-kube-api-access-vzwz2\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.894389 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.894346    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29" event={"ID":"bf4d7655-e2fc-4e5f-8675-0eb67b1d7461","Type":"ContainerDied","Data":"47114ff4b39abdd940f5adee455addb310da1c79d88a02869099c31277b03146"}
Sep 11 17:53:55.894389 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.894378    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"
Sep 11 17:53:55.894928 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.894420    2582 scope.go:117] "RemoveContainer" containerID="218e8e756088b331f96f55148307b6c39c22da5f74b0ef059860ea4fcde4563c"
Sep 11 17:53:55.895915 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.895880    2582 generic.go:356] "Generic (PLEG): container finished" podID="68c857a9-3cab-47a1-a15a-be0482ac0c87" containerID="ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351" exitCode=0
Sep 11 17:53:55.895982 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.895960    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"
Sep 11 17:53:55.896140 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.896111    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph" event={"ID":"68c857a9-3cab-47a1-a15a-be0482ac0c87","Type":"ContainerDied","Data":"ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351"}
Sep 11 17:53:55.896225 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.896175    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph" event={"ID":"68c857a9-3cab-47a1-a15a-be0482ac0c87","Type":"ContainerDied","Data":"61f679ea648ac354825a5fc59481a6c6b00e8d3eb20d7a3f9cfbe35e81ebadc1"}
Sep 11 17:53:55.898449 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.898421    2582 generic.go:356] "Generic (PLEG): container finished" podID="42814990-b4d0-47a7-a687-ef0a597e7f3d" containerID="79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448" exitCode=0
Sep 11 17:53:55.898588 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.898479    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"
Sep 11 17:53:55.898588 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.898545    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b" event={"ID":"42814990-b4d0-47a7-a687-ef0a597e7f3d","Type":"ContainerDied","Data":"79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448"}
Sep 11 17:53:55.898699 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.898587    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b" event={"ID":"42814990-b4d0-47a7-a687-ef0a597e7f3d","Type":"ContainerDied","Data":"427c824ceb643181dff3a9adf4282c979535f0e7a23669a09569149a83091bc4"}
Sep 11 17:53:55.905883 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.905719    2582 scope.go:117] "RemoveContainer" containerID="ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351"
Sep 11 17:53:55.918220 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.918193    2582 scope.go:117] "RemoveContainer" containerID="ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351"
Sep 11 17:53:55.918647 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:53:55.918619    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351\": container with ID starting with ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351 not found: ID does not exist" containerID="ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351"
Sep 11 17:53:55.918805 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.918659    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351"} err="failed to get container status \"ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351\": rpc error: code = NotFound desc = could not find container \"ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351\": container with ID starting with ef69586044a6d66d9b4f4d85d73b7c5a54184f2031a14108648f0b8f9e807351 not found: ID does not exist"
Sep 11 17:53:55.918805 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.918681    2582 scope.go:117] "RemoveContainer" containerID="79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448"
Sep 11 17:53:55.927740 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.927709    2582 scope.go:117] "RemoveContainer" containerID="79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448"
Sep 11 17:53:55.928245 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:53:55.928189    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448\": container with ID starting with 79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448 not found: ID does not exist" containerID="79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448"
Sep 11 17:53:55.928453 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.928241    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448"} err="failed to get container status \"79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448\": rpc error: code = NotFound desc = could not find container \"79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448\": container with ID starting with 79d6dab7470fa02e91aac65c3efe812193caabeea317c352edecea5082bcd448 not found: ID does not exist"
Sep 11 17:53:55.933774 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.933745    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"]
Sep 11 17:53:55.938987 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.938958    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-37899e-85977bfc46-788ph"]
Sep 11 17:53:55.958337 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.958257    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"]
Sep 11 17:53:55.977151 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.977099    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5f786284-546cb49cc7-tkr29"]
Sep 11 17:53:55.990198 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.990167    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"]
Sep 11 17:53:55.997985 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:55.997930    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-37899e-6fb64dc675-dwf5b"]
Sep 11 17:53:56.212473 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:56.212390    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:56.212473 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:56.212431    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:56.225052 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:56.225026    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:53:56.634141 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:56.634041    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42814990-b4d0-47a7-a687-ef0a597e7f3d" path="/var/lib/kubelet/pods/42814990-b4d0-47a7-a687-ef0a597e7f3d/volumes"
Sep 11 17:53:56.634485 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:56.634472    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68c857a9-3cab-47a1-a15a-be0482ac0c87" path="/var/lib/kubelet/pods/68c857a9-3cab-47a1-a15a-be0482ac0c87/volumes"
Sep 11 17:53:56.634791 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:56.634780    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf4d7655-e2fc-4e5f-8675-0eb67b1d7461" path="/var/lib/kubelet/pods/bf4d7655-e2fc-4e5f-8675-0eb67b1d7461/volumes"
Sep 11 17:53:56.919517 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:53:56.919488    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:54:00.530653 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:00.530615    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"]
Sep 11 17:54:15.550721 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.550674    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm" podUID="162fbab0-8e47-4c0e-8abe-23806ea01487" containerName="main" containerID="cri-o://92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016" gracePeriod=45
Sep 11 17:54:15.807411 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.807346    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:54:15.878405 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.878372    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/162fbab0-8e47-4c0e-8abe-23806ea01487-tls-certs\") pod \"162fbab0-8e47-4c0e-8abe-23806ea01487\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") "
Sep 11 17:54:15.878405 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.878412    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-dshm\") pod \"162fbab0-8e47-4c0e-8abe-23806ea01487\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") "
Sep 11 17:54:15.878695 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.878438    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd75t\" (UniqueName: \"kubernetes.io/projected/162fbab0-8e47-4c0e-8abe-23806ea01487-kube-api-access-pd75t\") pod \"162fbab0-8e47-4c0e-8abe-23806ea01487\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") "
Sep 11 17:54:15.878695 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.878493    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-tmp-dir\") pod \"162fbab0-8e47-4c0e-8abe-23806ea01487\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") "
Sep 11 17:54:15.878695 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.878526    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-home\") pod \"162fbab0-8e47-4c0e-8abe-23806ea01487\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") "
Sep 11 17:54:15.878884 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.878690    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-model-cache\") pod \"162fbab0-8e47-4c0e-8abe-23806ea01487\" (UID: \"162fbab0-8e47-4c0e-8abe-23806ea01487\") "
Sep 11 17:54:15.878884 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.878792    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "162fbab0-8e47-4c0e-8abe-23806ea01487" (UID: "162fbab0-8e47-4c0e-8abe-23806ea01487"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:15.878884 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.878811    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-home" (OuterVolumeSpecName: "home") pod "162fbab0-8e47-4c0e-8abe-23806ea01487" (UID: "162fbab0-8e47-4c0e-8abe-23806ea01487"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:15.879110 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.878925    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-model-cache" (OuterVolumeSpecName: "model-cache") pod "162fbab0-8e47-4c0e-8abe-23806ea01487" (UID: "162fbab0-8e47-4c0e-8abe-23806ea01487"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:15.879110 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.879070    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-tmp-dir\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:15.879394 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.879119    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-home\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:15.879394 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.879132    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-model-cache\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:15.880865 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.880839    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/162fbab0-8e47-4c0e-8abe-23806ea01487-kube-api-access-pd75t" (OuterVolumeSpecName: "kube-api-access-pd75t") pod "162fbab0-8e47-4c0e-8abe-23806ea01487" (UID: "162fbab0-8e47-4c0e-8abe-23806ea01487"). InnerVolumeSpecName "kube-api-access-pd75t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:54:15.880865 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.880861    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-dshm" (OuterVolumeSpecName: "dshm") pod "162fbab0-8e47-4c0e-8abe-23806ea01487" (UID: "162fbab0-8e47-4c0e-8abe-23806ea01487"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:15.881012 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.880870    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/162fbab0-8e47-4c0e-8abe-23806ea01487-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "162fbab0-8e47-4c0e-8abe-23806ea01487" (UID: "162fbab0-8e47-4c0e-8abe-23806ea01487"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:54:15.979941 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.979910    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/162fbab0-8e47-4c0e-8abe-23806ea01487-tls-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:15.979941 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.979938    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/162fbab0-8e47-4c0e-8abe-23806ea01487-dshm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:15.979941 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:15.979948    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd75t\" (UniqueName: \"kubernetes.io/projected/162fbab0-8e47-4c0e-8abe-23806ea01487-kube-api-access-pd75t\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:16.003738 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:16.003706    2582 generic.go:356] "Generic (PLEG): container finished" podID="162fbab0-8e47-4c0e-8abe-23806ea01487" containerID="92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016" exitCode=0
Sep 11 17:54:16.003887 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:16.003751    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"
Sep 11 17:54:16.003887 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:16.003778    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm" event={"ID":"162fbab0-8e47-4c0e-8abe-23806ea01487","Type":"ContainerDied","Data":"92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016"}
Sep 11 17:54:16.003887 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:16.003821    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm" event={"ID":"162fbab0-8e47-4c0e-8abe-23806ea01487","Type":"ContainerDied","Data":"3c93de1bf9f026423317047463e42dd50a2ad32aad3ddbd8e6f10ef23cc84a43"}
Sep 11 17:54:16.003887 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:16.003838    2582 scope.go:117] "RemoveContainer" containerID="92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016"
Sep 11 17:54:16.013373 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:16.013353    2582 scope.go:117] "RemoveContainer" containerID="92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016"
Sep 11 17:54:16.013627 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:54:16.013610    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016\": container with ID starting with 92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016 not found: ID does not exist" containerID="92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016"
Sep 11 17:54:16.013679 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:16.013635    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016"} err="failed to get container status \"92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016\": rpc error: code = NotFound desc = could not find container \"92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016\": container with ID starting with 92cf1ef65055317b752db580d0e1c45f57a5da9f8e17e532d230772e2ac49016 not found: ID does not exist"
Sep 11 17:54:16.025663 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:16.025626    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"]
Sep 11 17:54:16.029732 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:16.029702    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-37899e/autores-model-37899e-kserve-544dd4547-m2kzm"]
Sep 11 17:54:16.633760 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:16.633722    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="162fbab0-8e47-4c0e-8abe-23806ea01487" path="/var/lib/kubelet/pods/162fbab0-8e47-4c0e-8abe-23806ea01487/volumes"
Sep 11 17:54:33.791516 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791481    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"]
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791833    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="162fbab0-8e47-4c0e-8abe-23806ea01487" containerName="main"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791845    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="162fbab0-8e47-4c0e-8abe-23806ea01487" containerName="main"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791856    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42814990-b4d0-47a7-a687-ef0a597e7f3d" containerName="payload-processing"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791862    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="42814990-b4d0-47a7-a687-ef0a597e7f3d" containerName="payload-processing"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791868    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68c857a9-3cab-47a1-a15a-be0482ac0c87" containerName="maas-api"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791874    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c857a9-3cab-47a1-a15a-be0482ac0c87" containerName="maas-api"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791894    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4d7655-e2fc-4e5f-8675-0eb67b1d7461" containerName="payload-processing"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791899    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4d7655-e2fc-4e5f-8675-0eb67b1d7461" containerName="payload-processing"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791950    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68c857a9-3cab-47a1-a15a-be0482ac0c87" containerName="maas-api"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791958    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="42814990-b4d0-47a7-a687-ef0a597e7f3d" containerName="payload-processing"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791964    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="162fbab0-8e47-4c0e-8abe-23806ea01487" containerName="main"
Sep 11 17:54:33.791971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.791970    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf4d7655-e2fc-4e5f-8675-0eb67b1d7461" containerName="payload-processing"
Sep 11 17:54:33.794812 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.794787    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:33.797899 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.797878    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"premium-w0-286c27-kserve-self-signed-certs\""
Sep 11 17:54:33.810112 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.808815    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"]
Sep 11 17:54:33.956902 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.956858    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-model-cache\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:33.956902 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.956902    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2txwx\" (UniqueName: \"kubernetes.io/projected/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-kube-api-access-2txwx\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:33.957189 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.956979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tls-certs\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:33.957189 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.957033    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tmp-dir\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:33.957189 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.957074    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-dshm\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:33.957322 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:33.957180    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-home\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.058846 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.058736    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-model-cache\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.058846 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.058782    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2txwx\" (UniqueName: \"kubernetes.io/projected/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-kube-api-access-2txwx\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.058846 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.058814    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tls-certs\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.058846 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.058838    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tmp-dir\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.059246 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.058871    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-dshm\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.059246 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.058984    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-home\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.059357 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.059291    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-model-cache\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.059357 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.059307    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tmp-dir\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.059357 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.059349    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-home\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.061335 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.061317    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-dshm\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.061562 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.061544    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tls-certs\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.068332 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.068293    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2txwx\" (UniqueName: \"kubernetes.io/projected/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-kube-api-access-2txwx\") pod \"premium-w0-286c27-kserve-7f57df4c58-d9fpw\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") " pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.104499 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.104455    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:34.383746 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:34.383712    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"]
Sep 11 17:54:34.386391 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:54:34.386363    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42de3dc5_f6d7_48df_9fd1_9d8d6cdbc83b.slice/crio-d1c0f9216f214ee3386eba8608a10a0b202fdf0a5d55890e098c51f897eff19b WatchSource:0}: Error finding container d1c0f9216f214ee3386eba8608a10a0b202fdf0a5d55890e098c51f897eff19b: Status 404 returned error can't find the container with id d1c0f9216f214ee3386eba8608a10a0b202fdf0a5d55890e098c51f897eff19b
Sep 11 17:54:35.101478 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:35.101438    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw" event={"ID":"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b","Type":"ContainerStarted","Data":"4c8c6cc6fdcf245bed453c1833d5a655cf1c54509013ab4629cc8605b9fcc590"}
Sep 11 17:54:35.101478 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:35.101486    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw" event={"ID":"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b","Type":"ContainerStarted","Data":"d1c0f9216f214ee3386eba8608a10a0b202fdf0a5d55890e098c51f897eff19b"}
Sep 11 17:54:35.122689 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:35.122631    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw" podStartSLOduration=2.122613385 podStartE2EDuration="2.122613385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:54:35.11986973 +0000 UTC m=+943.008048077" watchObservedRunningTime="2026-09-11 17:54:35.122613385 +0000 UTC m=+943.010791737"
Sep 11 17:54:38.605442 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:38.605407    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"]
Sep 11 17:54:38.605937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:38.605615    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy" containerID="cri-o://0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea" gracePeriod=30
Sep 11 17:54:39.408186 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.408152    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"]
Sep 11 17:54:39.411835 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.411813    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.415015 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.414989    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-104872-openshift-default-dockercfg-pn959\""
Sep 11 17:54:39.428232 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.428204    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"]
Sep 11 17:54:39.512027 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.511979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-certs\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.512227 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.512034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istiod-ca-cert\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.512227 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.512117    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-socket\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.512227 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.512142    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-token\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.512227 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.512215    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwt88\" (UniqueName: \"kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-kube-api-access-bwt88\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.512488 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.512298    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-credential-socket\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.512488 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.512339    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-envoy\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.512488 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.512382    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-data\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.512488 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.512413    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-podinfo\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.613173 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.613134    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-socket\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.613657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.613183    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-token\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.613657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.613237    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwt88\" (UniqueName: \"kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-kube-api-access-bwt88\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.613657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.613303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-credential-socket\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.613657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.613336    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-envoy\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.613657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.613377    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-data\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.613657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.613409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-podinfo\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.613657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.613454    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-certs\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.613657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.613478    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istiod-ca-cert\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.613657 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.613616    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-socket\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.614147 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.614062    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-certs\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.614214 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.614198    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istiod-ca-cert\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.614295 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.614266    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-credential-socket\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.614490 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.614471    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-data\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.616216 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.616192    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-envoy\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.616712 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.616690    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-podinfo\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.624753 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.624724    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-token\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.624883 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.624790    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwt88\" (UniqueName: \"kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-kube-api-access-bwt88\") pod \"e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") " pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:39.725313 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:39.725227    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:40.011406 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:40.011377    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"]
Sep 11 17:54:40.013307 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:54:40.013269    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3b765f_5c9d_4cce_9a7d_61846ee013fb.slice/crio-55568cc59db837f5193beac9e1ad60d2dedf92f9ef0a91c7a08a656856c5610a WatchSource:0}: Error finding container 55568cc59db837f5193beac9e1ad60d2dedf92f9ef0a91c7a08a656856c5610a: Status 404 returned error can't find the container with id 55568cc59db837f5193beac9e1ad60d2dedf92f9ef0a91c7a08a656856c5610a
Sep 11 17:54:40.016128 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:40.016068    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:54:40.016248 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:40.016173    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:54:40.016248 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:40.016217    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:54:40.132536 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:40.132500    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" event={"ID":"ae3b765f-5c9d-4cce-9a7d-61846ee013fb","Type":"ContainerStarted","Data":"dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040"}
Sep 11 17:54:40.132720 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:40.132554    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" event={"ID":"ae3b765f-5c9d-4cce-9a7d-61846ee013fb","Type":"ContainerStarted","Data":"55568cc59db837f5193beac9e1ad60d2dedf92f9ef0a91c7a08a656856c5610a"}
Sep 11 17:54:40.161857 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:40.161788    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" podStartSLOduration=1.161763219 podStartE2EDuration="1.161763219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:54:40.161307336 +0000 UTC m=+948.049485683" watchObservedRunningTime="2026-09-11 17:54:40.161763219 +0000 UTC m=+948.049941565"
Sep 11 17:54:40.726203 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:40.726171    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:41.731139 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:41.731108    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:42.141849 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:42.141808    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:42.142691 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:42.142673    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:54:43.855181 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.855156    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:54:43.956297 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.956258    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-podinfo\") pod \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") "
Sep 11 17:54:43.956508 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.956307    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-token\") pod \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") "
Sep 11 17:54:43.956508 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.956360    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-credential-socket\") pod \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") "
Sep 11 17:54:43.956508 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.956398    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-envoy\") pod \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") "
Sep 11 17:54:43.956508 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.956431    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-data\") pod \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") "
Sep 11 17:54:43.956508 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.956475    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96h7h\" (UniqueName: \"kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-kube-api-access-96h7h\") pod \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") "
Sep 11 17:54:43.956766 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.956517    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istiod-ca-cert\") pod \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") "
Sep 11 17:54:43.956766 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.956555    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-certs\") pod \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") "
Sep 11 17:54:43.956766 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.956714    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f56fc9f8-bfde-4e98-9b4c-cf04170b620d" (UID: "f56fc9f8-bfde-4e98-9b4c-cf04170b620d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:43.956960 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.956775    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-socket\") pod \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\" (UID: \"f56fc9f8-bfde-4e98-9b4c-cf04170b620d\") "
Sep 11 17:54:43.957174 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.957052    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f56fc9f8-bfde-4e98-9b4c-cf04170b620d" (UID: "f56fc9f8-bfde-4e98-9b4c-cf04170b620d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:54:43.957354 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.957186    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-credential-socket\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:43.957427 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.957347    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f56fc9f8-bfde-4e98-9b4c-cf04170b620d" (UID: "f56fc9f8-bfde-4e98-9b4c-cf04170b620d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:43.957427 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.957293    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-data" (OuterVolumeSpecName: "istio-data") pod "f56fc9f8-bfde-4e98-9b4c-cf04170b620d" (UID: "f56fc9f8-bfde-4e98-9b4c-cf04170b620d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:43.957529 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.957417    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f56fc9f8-bfde-4e98-9b4c-cf04170b620d" (UID: "f56fc9f8-bfde-4e98-9b4c-cf04170b620d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:43.958750 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.958729    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-token" (OuterVolumeSpecName: "istio-token") pod "f56fc9f8-bfde-4e98-9b4c-cf04170b620d" (UID: "f56fc9f8-bfde-4e98-9b4c-cf04170b620d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:54:43.958837 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.958792    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f56fc9f8-bfde-4e98-9b4c-cf04170b620d" (UID: "f56fc9f8-bfde-4e98-9b4c-cf04170b620d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:54:43.959320 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.959300    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f56fc9f8-bfde-4e98-9b4c-cf04170b620d" (UID: "f56fc9f8-bfde-4e98-9b4c-cf04170b620d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:43.959320 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:43.959307    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-kube-api-access-96h7h" (OuterVolumeSpecName: "kube-api-access-96h7h") pod "f56fc9f8-bfde-4e98-9b4c-cf04170b620d" (UID: "f56fc9f8-bfde-4e98-9b4c-cf04170b620d"). InnerVolumeSpecName "kube-api-access-96h7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:54:44.058699 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.058657    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-envoy\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:44.058699 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.058690    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-data\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:44.058699 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.058703    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96h7h\" (UniqueName: \"kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-kube-api-access-96h7h\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:44.058965 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.058715    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istiod-ca-cert\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:44.058965 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.058726    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:44.058965 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.058737    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-workload-socket\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:44.058965 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.058749    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-podinfo\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:44.058965 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.058760    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f56fc9f8-bfde-4e98-9b4c-cf04170b620d-istio-token\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:54:44.105627 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.105583    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:44.105813 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.105646    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:44.118734 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.118702    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:44.152309 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.152281    2582 generic.go:356] "Generic (PLEG): container finished" podID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerID="0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea" exitCode=0
Sep 11 17:54:44.152495 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.152336    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"
Sep 11 17:54:44.152495 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.152375    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" event={"ID":"f56fc9f8-bfde-4e98-9b4c-cf04170b620d","Type":"ContainerDied","Data":"0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea"}
Sep 11 17:54:44.152495 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.152430    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn" event={"ID":"f56fc9f8-bfde-4e98-9b4c-cf04170b620d","Type":"ContainerDied","Data":"b9fad1e7aa70612dda67c6f0a20523a7ed9385167d3fef990d4789e5a943638e"}
Sep 11 17:54:44.152495 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.152451    2582 scope.go:117] "RemoveContainer" containerID="0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea"
Sep 11 17:54:44.163259 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.163241    2582 scope.go:117] "RemoveContainer" containerID="c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f"
Sep 11 17:54:44.166505 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.166484    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 17:54:44.171828 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.171811    2582 scope.go:117] "RemoveContainer" containerID="0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea"
Sep 11 17:54:44.172134 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:54:44.172115    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea\": container with ID starting with 0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea not found: ID does not exist" containerID="0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea"
Sep 11 17:54:44.172183 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.172145    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea"} err="failed to get container status \"0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea\": rpc error: code = NotFound desc = could not find container \"0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea\": container with ID starting with 0f39c5b1ed095392f645d5c19cbef0c5e8571a234d489a0de3628ebb168bc9ea not found: ID does not exist"
Sep 11 17:54:44.172183 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.172165    2582 scope.go:117] "RemoveContainer" containerID="c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f"
Sep 11 17:54:44.172427 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:54:44.172410    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f\": container with ID starting with c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f not found: ID does not exist" containerID="c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f"
Sep 11 17:54:44.172464 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.172434    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f"} err="failed to get container status \"c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f\": rpc error: code = NotFound desc = could not find container \"c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f\": container with ID starting with c0571b8cbfa9718a6b11f9e2c8d1f221e850245751b4782de4f1efa08e1c556f not found: ID does not exist"
Sep 11 17:54:44.183546 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.183517    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"]
Sep 11 17:54:44.188430 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.188405    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-37899e-openshift-default-7cf8f5895b-gg9wn"]
Sep 11 17:54:44.634183 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:44.634149    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" path="/var/lib/kubelet/pods/f56fc9f8-bfde-4e98-9b4c-cf04170b620d/volumes"
Sep 11 17:54:45.788201 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.788161    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"]
Sep 11 17:54:45.788741 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.788722    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy"
Sep 11 17:54:45.788839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.788743    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy"
Sep 11 17:54:45.788839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.788769    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy"
Sep 11 17:54:45.788839 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.788777    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy"
Sep 11 17:54:45.788994 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.788864    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy"
Sep 11 17:54:45.793558 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.793536    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.796734 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.796709    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:54:45.797001 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.796987    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5vrxn\""
Sep 11 17:54:45.797889 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.797873    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 17:54:45.797964 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.797922    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-104872-serving-cert\""
Sep 11 17:54:45.798038 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.798028    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 17:54:45.819715 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.819688    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"]
Sep 11 17:54:45.875938 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.875901    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-maas-api-tls\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.876143 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.875965    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ht8p\" (UniqueName: \"kubernetes.io/projected/2f501598-617a-4ff9-a9d9-f1231ee06193-kube-api-access-9ht8p\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.876143 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.876006    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-metrics-certs\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.876143 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.876116    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f501598-617a-4ff9-a9d9-f1231ee06193-openshift-service-ca\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.977450 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.977418    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f501598-617a-4ff9-a9d9-f1231ee06193-openshift-service-ca\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.977688 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.977493    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-maas-api-tls\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.977688 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.977536    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ht8p\" (UniqueName: \"kubernetes.io/projected/2f501598-617a-4ff9-a9d9-f1231ee06193-kube-api-access-9ht8p\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.977688 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.977568    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-metrics-certs\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.977688 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:54:45.977651    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-104872-serving-cert: secret "maas-api-e2e-explicit-104872-serving-cert" not found
Sep 11 17:54:45.977918 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:54:45.977742    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-maas-api-tls podName:2f501598-617a-4ff9-a9d9-f1231ee06193 nodeName:}" failed. No retries permitted until 2026-09-11 17:54:46.477718604 +0000 UTC m=+954.365896930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-maas-api-tls") pod "maas-api-e2e-explicit-104872-6498c7f64b-cxsd6" (UID: "2f501598-617a-4ff9-a9d9-f1231ee06193") : secret "maas-api-e2e-explicit-104872-serving-cert" not found
Sep 11 17:54:45.978189 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.978169    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f501598-617a-4ff9-a9d9-f1231ee06193-openshift-service-ca\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.980126 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.980106    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-metrics-certs\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:45.998793 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:45.998764    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ht8p\" (UniqueName: \"kubernetes.io/projected/2f501598-617a-4ff9-a9d9-f1231ee06193-kube-api-access-9ht8p\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:46.483614 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.483575    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-maas-api-tls\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:46.486125 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.486102    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-maas-api-tls\") pod \"maas-api-e2e-explicit-104872-6498c7f64b-cxsd6\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:46.703056 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.703024    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:46.769310 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.769271    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"]
Sep 11 17:54:46.770220 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.770196    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f56fc9f8-bfde-4e98-9b4c-cf04170b620d" containerName="istio-proxy"
Sep 11 17:54:46.776648 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.775846    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:54:46.781680 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.781532    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-104872-dockercfg-jvsfr\""
Sep 11 17:54:46.782367 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.782336    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-104872\""
Sep 11 17:54:46.786112 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.785907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c4b6e4-af79-4e75-a422-60df44dfa18e-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg\" (UID: \"51c4b6e4-af79-4e75-a422-60df44dfa18e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:54:46.786112 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.785983    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flvvm\" (UniqueName: \"kubernetes.io/projected/51c4b6e4-af79-4e75-a422-60df44dfa18e-kube-api-access-flvvm\") pod \"payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg\" (UID: \"51c4b6e4-af79-4e75-a422-60df44dfa18e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:54:46.787042 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.786972    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"]
Sep 11 17:54:46.886713 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.886681    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flvvm\" (UniqueName: \"kubernetes.io/projected/51c4b6e4-af79-4e75-a422-60df44dfa18e-kube-api-access-flvvm\") pod \"payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg\" (UID: \"51c4b6e4-af79-4e75-a422-60df44dfa18e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:54:46.887149 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.886777    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c4b6e4-af79-4e75-a422-60df44dfa18e-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg\" (UID: \"51c4b6e4-af79-4e75-a422-60df44dfa18e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:54:46.887538 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.887517    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c4b6e4-af79-4e75-a422-60df44dfa18e-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg\" (UID: \"51c4b6e4-af79-4e75-a422-60df44dfa18e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:54:46.896350 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.896326    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flvvm\" (UniqueName: \"kubernetes.io/projected/51c4b6e4-af79-4e75-a422-60df44dfa18e-kube-api-access-flvvm\") pod \"payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg\" (UID: \"51c4b6e4-af79-4e75-a422-60df44dfa18e\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:54:46.989334 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.989304    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"]
Sep 11 17:54:46.990376 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:54:46.990353    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f501598_617a_4ff9_a9d9_f1231ee06193.slice/crio-19a9e3695053a8a9f64745e59bc7ff30dbaed9c9038df39efb2f2a993f9d778f WatchSource:0}: Error finding container 19a9e3695053a8a9f64745e59bc7ff30dbaed9c9038df39efb2f2a993f9d778f: Status 404 returned error can't find the container with id 19a9e3695053a8a9f64745e59bc7ff30dbaed9c9038df39efb2f2a993f9d778f
Sep 11 17:54:46.991808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:46.991790    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:54:47.090449 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:47.090343    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:54:47.170698 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:47.170659    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6" event={"ID":"2f501598-617a-4ff9-a9d9-f1231ee06193","Type":"ContainerStarted","Data":"19a9e3695053a8a9f64745e59bc7ff30dbaed9c9038df39efb2f2a993f9d778f"}
Sep 11 17:54:47.371604 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:47.371580    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"]
Sep 11 17:54:47.373296 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:54:47.373273    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51c4b6e4_af79_4e75_a422_60df44dfa18e.slice/crio-b8b31e1e617d19ca1eedb874aecb8bc2358b58fcd24aadcdd6ba66f265da1f6c WatchSource:0}: Error finding container b8b31e1e617d19ca1eedb874aecb8bc2358b58fcd24aadcdd6ba66f265da1f6c: Status 404 returned error can't find the container with id b8b31e1e617d19ca1eedb874aecb8bc2358b58fcd24aadcdd6ba66f265da1f6c
Sep 11 17:54:48.177500 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:48.177462    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6" event={"ID":"2f501598-617a-4ff9-a9d9-f1231ee06193","Type":"ContainerStarted","Data":"3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a"}
Sep 11 17:54:48.177972 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:48.177632    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:48.179137 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:48.179106    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg" event={"ID":"51c4b6e4-af79-4e75-a422-60df44dfa18e","Type":"ContainerStarted","Data":"0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7"}
Sep 11 17:54:48.179260 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:48.179155    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:54:48.179260 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:48.179180    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg" event={"ID":"51c4b6e4-af79-4e75-a422-60df44dfa18e","Type":"ContainerStarted","Data":"b8b31e1e617d19ca1eedb874aecb8bc2358b58fcd24aadcdd6ba66f265da1f6c"}
Sep 11 17:54:48.197331 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:48.197274    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6" podStartSLOduration=2.852039528 podStartE2EDuration="3.197257224s" totalImagesPullingTime="345.217696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:54:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:54:48.195475472 +0000 UTC m=+956.083653820" watchObservedRunningTime="2026-09-11 17:54:48.197257224 +0000 UTC m=+956.085435569"
Sep 11 17:54:48.213689 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:48.213638    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg" podStartSLOduration=2.213623273 podStartE2EDuration="2.213623273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:54:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:54:48.212506978 +0000 UTC m=+956.100685325" watchObservedRunningTime="2026-09-11 17:54:48.213623273 +0000 UTC m=+956.101801666"
Sep 11 17:54:54.190746 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:54.190712    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:54:59.186431 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:54:59.186395    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:55:05.281690 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:05.281656    2582 generic.go:356] "Generic (PLEG): container finished" podID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerID="dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040" exitCode=0
Sep 11 17:55:05.282213 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:05.281735    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" event={"ID":"ae3b765f-5c9d-4cce-9a7d-61846ee013fb","Type":"ContainerDied","Data":"dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040"}
Sep 11 17:55:05.282213 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:05.282174    2582 scope.go:117] "RemoveContainer" containerID="dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040"
Sep 11 17:55:05.283349 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:05.283317    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:55:05.283437 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:05.283385    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:55:05.283437 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:05.283422    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 11 17:55:06.288557 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:06.288523    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" event={"ID":"ae3b765f-5c9d-4cce-9a7d-61846ee013fb","Type":"ContainerStarted","Data":"8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22"}
Sep 11 17:55:06.726665 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:06.726621    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:55:06.726905 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:06.726751    2582 patch_prober.go:28] interesting pod/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 11 17:55:06.726905 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:06.726822    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 11 17:55:07.726478 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:07.726439    2582 patch_prober.go:28] interesting pod/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 11 17:55:07.727107 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:07.727049    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 11 17:55:08.725811 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:08.725761    2582 patch_prober.go:28] interesting pod/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 11 17:55:08.726037 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:08.725858    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 11 17:55:09.726575 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:09.726524    2582 patch_prober.go:28] interesting pod/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 11 17:55:09.727134 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:09.726617    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 11 17:55:10.726642 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:10.726602    2582 patch_prober.go:28] interesting pod/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 11 17:55:10.727284 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:10.727253    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 11 17:55:11.730546 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:11.730514    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:55:11.730946 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:11.730806    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:55:11.731636 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:11.731617    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:55:44.518267 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.518222    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"]
Sep 11 17:55:44.521360 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.521337    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:55:44.524772 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.524746    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-fcdaf27d\""
Sep 11 17:55:44.532653 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.532623    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"]
Sep 11 17:55:44.611153 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.611111    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abdc631e-8dcd-4baf-a70f-07e96afebea6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n\" (UID: \"abdc631e-8dcd-4baf-a70f-07e96afebea6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:55:44.611335 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.611168    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6lqk\" (UniqueName: \"kubernetes.io/projected/abdc631e-8dcd-4baf-a70f-07e96afebea6-kube-api-access-s6lqk\") pod \"payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n\" (UID: \"abdc631e-8dcd-4baf-a70f-07e96afebea6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:55:44.633963 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.633934    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"]
Sep 11 17:55:44.636636 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.636620    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:55:44.639864 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.639838    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-fcdaf27d-dockercfg-p8kf2\""
Sep 11 17:55:44.643927 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.643898    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"]
Sep 11 17:55:44.712060 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.712019    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6lqk\" (UniqueName: \"kubernetes.io/projected/abdc631e-8dcd-4baf-a70f-07e96afebea6-kube-api-access-s6lqk\") pod \"payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n\" (UID: \"abdc631e-8dcd-4baf-a70f-07e96afebea6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:55:44.712274 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.712147    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-plugins-config-volume\") pod \"payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57\" (UID: \"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa\") " pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:55:44.712274 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.712192    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abdc631e-8dcd-4baf-a70f-07e96afebea6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n\" (UID: \"abdc631e-8dcd-4baf-a70f-07e96afebea6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:55:44.712274 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.712213    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm9l8\" (UniqueName: \"kubernetes.io/projected/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-kube-api-access-jm9l8\") pod \"payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57\" (UID: \"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa\") " pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:55:44.712937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.712919    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abdc631e-8dcd-4baf-a70f-07e96afebea6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n\" (UID: \"abdc631e-8dcd-4baf-a70f-07e96afebea6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:55:44.723787 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.723761    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6lqk\" (UniqueName: \"kubernetes.io/projected/abdc631e-8dcd-4baf-a70f-07e96afebea6-kube-api-access-s6lqk\") pod \"payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n\" (UID: \"abdc631e-8dcd-4baf-a70f-07e96afebea6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:55:44.812830 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.812738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm9l8\" (UniqueName: \"kubernetes.io/projected/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-kube-api-access-jm9l8\") pod \"payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57\" (UID: \"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa\") " pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:55:44.812991 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.812841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-plugins-config-volume\") pod \"payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57\" (UID: \"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa\") " pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:55:44.813538 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.813520    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-plugins-config-volume\") pod \"payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57\" (UID: \"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa\") " pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:55:44.822791 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.822760    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm9l8\" (UniqueName: \"kubernetes.io/projected/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-kube-api-access-jm9l8\") pod \"payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57\" (UID: \"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa\") " pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:55:44.832497 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.832465    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:55:44.949010 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:44.948972    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:55:45.128152 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:45.128118    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"]
Sep 11 17:55:45.129790 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:55:45.129762    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabdc631e_8dcd_4baf_a70f_07e96afebea6.slice/crio-8b369ffa08bf20876b67a6437b97c033550a2dc005e46bfb91d11ef3adf6a1fd WatchSource:0}: Error finding container 8b369ffa08bf20876b67a6437b97c033550a2dc005e46bfb91d11ef3adf6a1fd: Status 404 returned error can't find the container with id 8b369ffa08bf20876b67a6437b97c033550a2dc005e46bfb91d11ef3adf6a1fd
Sep 11 17:55:45.227733 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:45.227707    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"]
Sep 11 17:55:45.230356 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:55:45.230322    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53414de3_5bbd_4b32_b2b2_b7a9e7ec58aa.slice/crio-820c6b17bfb7bc306faafe6d202834df32ea15415f4717e96e66b5c339b2d712 WatchSource:0}: Error finding container 820c6b17bfb7bc306faafe6d202834df32ea15415f4717e96e66b5c339b2d712: Status 404 returned error can't find the container with id 820c6b17bfb7bc306faafe6d202834df32ea15415f4717e96e66b5c339b2d712
Sep 11 17:55:45.485941 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:45.485900    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n" event={"ID":"abdc631e-8dcd-4baf-a70f-07e96afebea6","Type":"ContainerStarted","Data":"4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e"}
Sep 11 17:55:45.486165 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:45.485961    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n" event={"ID":"abdc631e-8dcd-4baf-a70f-07e96afebea6","Type":"ContainerStarted","Data":"8b369ffa08bf20876b67a6437b97c033550a2dc005e46bfb91d11ef3adf6a1fd"}
Sep 11 17:55:45.486165 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:45.486011    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:55:45.487717 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:45.487690    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57" event={"ID":"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa","Type":"ContainerStarted","Data":"983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196"}
Sep 11 17:55:45.487848 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:45.487735    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57" event={"ID":"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa","Type":"ContainerStarted","Data":"820c6b17bfb7bc306faafe6d202834df32ea15415f4717e96e66b5c339b2d712"}
Sep 11 17:55:45.487848 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:45.487778    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:55:45.508975 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:45.508917    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n" podStartSLOduration=1.508899982 podStartE2EDuration="1.508899982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:55:45.506789164 +0000 UTC m=+1013.394967501" watchObservedRunningTime="2026-09-11 17:55:45.508899982 +0000 UTC m=+1013.397078327"
Sep 11 17:55:45.526521 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:45.526470    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57" podStartSLOduration=1.526450726 podStartE2EDuration="1.526450726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:55:45.525117054 +0000 UTC m=+1013.413295404" watchObservedRunningTime="2026-09-11 17:55:45.526450726 +0000 UTC m=+1013.414629072"
Sep 11 17:55:56.496227 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:56.496195    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:55:56.496667 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:55:56.496263    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:56:26.078707 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.078664    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"]
Sep 11 17:56:26.082545 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.082527    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.091247 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.091211    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"kube-root-ca.crt\""
Sep 11 17:56:26.091671 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.091265    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"default-dockercfg-8n74d\""
Sep 11 17:56:26.091840 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.091389    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"e2e-conflict-a-2025bfc2-kserve-self-signed-certs\""
Sep 11 17:56:26.092514 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.092481    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"openshift-service-ca.crt\""
Sep 11 17:56:26.096368 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.096345    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"]
Sep 11 17:56:26.114890 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.114848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-tmp-dir\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.115104 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.114907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-home\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.115104 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.114995    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbkkw\" (UniqueName: \"kubernetes.io/projected/8301717f-ce6a-4edc-b17d-208645010ef0-kube-api-access-pbkkw\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.115104 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.115025    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-model-cache\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.115252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.115164    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8301717f-ce6a-4edc-b17d-208645010ef0-tls-certs\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.115252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.115203    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-dshm\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.216172 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.216140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8301717f-ce6a-4edc-b17d-208645010ef0-tls-certs\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.216349 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.216195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-dshm\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.216349 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.216238    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-tmp-dir\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.216349 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.216281    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-home\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.216349 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.216332    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbkkw\" (UniqueName: \"kubernetes.io/projected/8301717f-ce6a-4edc-b17d-208645010ef0-kube-api-access-pbkkw\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.216539 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.216358    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-model-cache\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.216743 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.216719    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-model-cache\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.216950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.216904    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-home\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.216950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.216914    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-tmp-dir\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.219107 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.219068    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-dshm\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.219107 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.219069    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8301717f-ce6a-4edc-b17d-208645010ef0-tls-certs\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.225662 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.225637    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbkkw\" (UniqueName: \"kubernetes.io/projected/8301717f-ce6a-4edc-b17d-208645010ef0-kube-api-access-pbkkw\") pod \"e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.397762 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.397730    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:26.671493 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.671466    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"]
Sep 11 17:56:26.675427 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:56:26.675386    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8301717f_ce6a_4edc_b17d_208645010ef0.slice/crio-1db402d9aaac40193eb7cd7d9a43fda2d0b3b078c510dc6c0c6a420d716b8b24 WatchSource:0}: Error finding container 1db402d9aaac40193eb7cd7d9a43fda2d0b3b078c510dc6c0c6a420d716b8b24: Status 404 returned error can't find the container with id 1db402d9aaac40193eb7cd7d9a43fda2d0b3b078c510dc6c0c6a420d716b8b24
Sep 11 17:56:26.698565 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:26.698532    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4" event={"ID":"8301717f-ce6a-4edc-b17d-208645010ef0","Type":"ContainerStarted","Data":"1db402d9aaac40193eb7cd7d9a43fda2d0b3b078c510dc6c0c6a420d716b8b24"}
Sep 11 17:56:27.706433 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:27.706396    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4" event={"ID":"8301717f-ce6a-4edc-b17d-208645010ef0","Type":"ContainerStarted","Data":"39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855"}
Sep 11 17:56:27.725302 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:27.725236    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4" podStartSLOduration=1.725213036 podStartE2EDuration="1.725213036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:56:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:56:27.724947796 +0000 UTC m=+1055.613126146" watchObservedRunningTime="2026-09-11 17:56:27.725213036 +0000 UTC m=+1055.613391382"
Sep 11 17:56:28.292432 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.292389    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"]
Sep 11 17:56:28.292844 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.292659    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57" podUID="53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa" containerName="payload-processing" containerID="cri-o://983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196" gracePeriod=30
Sep 11 17:56:28.303008 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.301212    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"]
Sep 11 17:56:28.303008 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.301817    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n" podUID="abdc631e-8dcd-4baf-a70f-07e96afebea6" containerName="payload-pre-processing" containerID="cri-o://4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e" gracePeriod=30
Sep 11 17:56:28.598639 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.598614    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:56:28.601951 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.601924    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:56:28.642240 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.642028    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm9l8\" (UniqueName: \"kubernetes.io/projected/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-kube-api-access-jm9l8\") pod \"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa\" (UID: \"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa\") "
Sep 11 17:56:28.642240 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.642210    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abdc631e-8dcd-4baf-a70f-07e96afebea6-plugins-config-volume\") pod \"abdc631e-8dcd-4baf-a70f-07e96afebea6\" (UID: \"abdc631e-8dcd-4baf-a70f-07e96afebea6\") "
Sep 11 17:56:28.642240 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.642239    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6lqk\" (UniqueName: \"kubernetes.io/projected/abdc631e-8dcd-4baf-a70f-07e96afebea6-kube-api-access-s6lqk\") pod \"abdc631e-8dcd-4baf-a70f-07e96afebea6\" (UID: \"abdc631e-8dcd-4baf-a70f-07e96afebea6\") "
Sep 11 17:56:28.642561 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.642274    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-plugins-config-volume\") pod \"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa\" (UID: \"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa\") "
Sep 11 17:56:28.643366 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.642941    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abdc631e-8dcd-4baf-a70f-07e96afebea6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abdc631e-8dcd-4baf-a70f-07e96afebea6" (UID: "abdc631e-8dcd-4baf-a70f-07e96afebea6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:56:28.643908 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.643868    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa" (UID: "53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:56:28.646326 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.646276    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abdc631e-8dcd-4baf-a70f-07e96afebea6-kube-api-access-s6lqk" (OuterVolumeSpecName: "kube-api-access-s6lqk") pod "abdc631e-8dcd-4baf-a70f-07e96afebea6" (UID: "abdc631e-8dcd-4baf-a70f-07e96afebea6"). InnerVolumeSpecName "kube-api-access-s6lqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:56:28.646452 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.646419    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-kube-api-access-jm9l8" (OuterVolumeSpecName: "kube-api-access-jm9l8") pod "53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa" (UID: "53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa"). InnerVolumeSpecName "kube-api-access-jm9l8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:56:28.714012 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.713974    2582 generic.go:356] "Generic (PLEG): container finished" podID="abdc631e-8dcd-4baf-a70f-07e96afebea6" containerID="4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e" exitCode=0
Sep 11 17:56:28.714485 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.714023    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"
Sep 11 17:56:28.714485 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.714091    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n" event={"ID":"abdc631e-8dcd-4baf-a70f-07e96afebea6","Type":"ContainerDied","Data":"4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e"}
Sep 11 17:56:28.714485 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.714153    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n" event={"ID":"abdc631e-8dcd-4baf-a70f-07e96afebea6","Type":"ContainerDied","Data":"8b369ffa08bf20876b67a6437b97c033550a2dc005e46bfb91d11ef3adf6a1fd"}
Sep 11 17:56:28.714485 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.714186    2582 scope.go:117] "RemoveContainer" containerID="4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e"
Sep 11 17:56:28.716702 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.716674    2582 generic.go:356] "Generic (PLEG): container finished" podID="53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa" containerID="983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196" exitCode=0
Sep 11 17:56:28.716851 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.716735    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"
Sep 11 17:56:28.716851 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.716800    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57" event={"ID":"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa","Type":"ContainerDied","Data":"983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196"}
Sep 11 17:56:28.716851 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.716846    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57" event={"ID":"53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa","Type":"ContainerDied","Data":"820c6b17bfb7bc306faafe6d202834df32ea15415f4717e96e66b5c339b2d712"}
Sep 11 17:56:28.724733 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.724706    2582 scope.go:117] "RemoveContainer" containerID="4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e"
Sep 11 17:56:28.725206 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:56:28.725183    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e\": container with ID starting with 4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e not found: ID does not exist" containerID="4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e"
Sep 11 17:56:28.725290 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.725221    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e"} err="failed to get container status \"4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e\": rpc error: code = NotFound desc = could not find container \"4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e\": container with ID starting with 4b214dabb3b5ed7bdad9ac9860ba39fbdd00266eb87ba1465d31abd759e04c0e not found: ID does not exist"
Sep 11 17:56:28.725290 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.725240    2582 scope.go:117] "RemoveContainer" containerID="983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196"
Sep 11 17:56:28.734724 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.734703    2582 scope.go:117] "RemoveContainer" containerID="983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196"
Sep 11 17:56:28.735106 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:56:28.735065    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196\": container with ID starting with 983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196 not found: ID does not exist" containerID="983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196"
Sep 11 17:56:28.735175 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.735115    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196"} err="failed to get container status \"983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196\": rpc error: code = NotFound desc = could not find container \"983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196\": container with ID starting with 983e0f684ae2c5c126eed0699fc3751956e684e6729a5583ce8e21e3480e9196 not found: ID does not exist"
Sep 11 17:56:28.743752 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.743725    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abdc631e-8dcd-4baf-a70f-07e96afebea6-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:28.743752 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.743753    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6lqk\" (UniqueName: \"kubernetes.io/projected/abdc631e-8dcd-4baf-a70f-07e96afebea6-kube-api-access-s6lqk\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:28.743884 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.743764    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:28.743884 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.743773    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm9l8\" (UniqueName: \"kubernetes.io/projected/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa-kube-api-access-jm9l8\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:28.751232 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.751192    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"]
Sep 11 17:56:28.772822 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.772791    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcdaf27d-549c7bfcb5-7xx57"]
Sep 11 17:56:28.792599 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.792556    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"]
Sep 11 17:56:28.804511 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:28.804476    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcdaf27d-fcbc7585d-sbx9n"]
Sep 11 17:56:30.634733 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:30.634692    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa" path="/var/lib/kubelet/pods/53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa/volumes"
Sep 11 17:56:30.635250 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:30.635233    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abdc631e-8dcd-4baf-a70f-07e96afebea6" path="/var/lib/kubelet/pods/abdc631e-8dcd-4baf-a70f-07e96afebea6/volumes"
Sep 11 17:56:36.398215 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:36.398168    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:36.398615 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:36.398255    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:36.411037 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:36.411012    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:36.769726 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:36.769636    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 17:56:41.801557 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:41.801516    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"]
Sep 11 17:56:41.801970 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:41.801789    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6" podUID="2f501598-617a-4ff9-a9d9-f1231ee06193" containerName="maas-api" containerID="cri-o://3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a" gracePeriod=30
Sep 11 17:56:41.867795 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:41.867760    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"]
Sep 11 17:56:41.868040 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:41.868015    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg" podUID="51c4b6e4-af79-4e75-a422-60df44dfa18e" containerName="payload-pre-processing" containerID="cri-o://0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7" gracePeriod=30
Sep 11 17:56:42.120846 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.120815    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:56:42.152458 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.152428    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:56:42.274178 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.274148    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-metrics-certs\") pod \"2f501598-617a-4ff9-a9d9-f1231ee06193\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") "
Sep 11 17:56:42.274369 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.274206    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-maas-api-tls\") pod \"2f501598-617a-4ff9-a9d9-f1231ee06193\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") "
Sep 11 17:56:42.274369 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.274254    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f501598-617a-4ff9-a9d9-f1231ee06193-openshift-service-ca\") pod \"2f501598-617a-4ff9-a9d9-f1231ee06193\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") "
Sep 11 17:56:42.274369 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.274275    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ht8p\" (UniqueName: \"kubernetes.io/projected/2f501598-617a-4ff9-a9d9-f1231ee06193-kube-api-access-9ht8p\") pod \"2f501598-617a-4ff9-a9d9-f1231ee06193\" (UID: \"2f501598-617a-4ff9-a9d9-f1231ee06193\") "
Sep 11 17:56:42.274369 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.274324    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c4b6e4-af79-4e75-a422-60df44dfa18e-plugins-config-volume\") pod \"51c4b6e4-af79-4e75-a422-60df44dfa18e\" (UID: \"51c4b6e4-af79-4e75-a422-60df44dfa18e\") "
Sep 11 17:56:42.274588 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.274389    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flvvm\" (UniqueName: \"kubernetes.io/projected/51c4b6e4-af79-4e75-a422-60df44dfa18e-kube-api-access-flvvm\") pod \"51c4b6e4-af79-4e75-a422-60df44dfa18e\" (UID: \"51c4b6e4-af79-4e75-a422-60df44dfa18e\") "
Sep 11 17:56:42.274739 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.274684    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f501598-617a-4ff9-a9d9-f1231ee06193-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f501598-617a-4ff9-a9d9-f1231ee06193" (UID: "2f501598-617a-4ff9-a9d9-f1231ee06193"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:56:42.274835 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.274734    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51c4b6e4-af79-4e75-a422-60df44dfa18e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "51c4b6e4-af79-4e75-a422-60df44dfa18e" (UID: "51c4b6e4-af79-4e75-a422-60df44dfa18e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:56:42.276781 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.276752    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f501598-617a-4ff9-a9d9-f1231ee06193-kube-api-access-9ht8p" (OuterVolumeSpecName: "kube-api-access-9ht8p") pod "2f501598-617a-4ff9-a9d9-f1231ee06193" (UID: "2f501598-617a-4ff9-a9d9-f1231ee06193"). InnerVolumeSpecName "kube-api-access-9ht8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:56:42.277031 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.276977    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51c4b6e4-af79-4e75-a422-60df44dfa18e-kube-api-access-flvvm" (OuterVolumeSpecName: "kube-api-access-flvvm") pod "51c4b6e4-af79-4e75-a422-60df44dfa18e" (UID: "51c4b6e4-af79-4e75-a422-60df44dfa18e"). InnerVolumeSpecName "kube-api-access-flvvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:56:42.277031 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.277008    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f501598-617a-4ff9-a9d9-f1231ee06193" (UID: "2f501598-617a-4ff9-a9d9-f1231ee06193"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:56:42.277189 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.277051    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2f501598-617a-4ff9-a9d9-f1231ee06193" (UID: "2f501598-617a-4ff9-a9d9-f1231ee06193"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:56:42.375666 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.375575    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-maas-api-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:42.375666 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.375610    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f501598-617a-4ff9-a9d9-f1231ee06193-openshift-service-ca\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:42.375666 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.375654    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ht8p\" (UniqueName: \"kubernetes.io/projected/2f501598-617a-4ff9-a9d9-f1231ee06193-kube-api-access-9ht8p\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:42.375889 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.375669    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c4b6e4-af79-4e75-a422-60df44dfa18e-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:42.375889 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.375682    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flvvm\" (UniqueName: \"kubernetes.io/projected/51c4b6e4-af79-4e75-a422-60df44dfa18e-kube-api-access-flvvm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:42.375889 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.375693    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f501598-617a-4ff9-a9d9-f1231ee06193-metrics-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:42.787755 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.787716    2582 generic.go:356] "Generic (PLEG): container finished" podID="2f501598-617a-4ff9-a9d9-f1231ee06193" containerID="3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a" exitCode=0
Sep 11 17:56:42.787929 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.787766    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"
Sep 11 17:56:42.787929 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.787798    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6" event={"ID":"2f501598-617a-4ff9-a9d9-f1231ee06193","Type":"ContainerDied","Data":"3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a"}
Sep 11 17:56:42.787929 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.787847    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6" event={"ID":"2f501598-617a-4ff9-a9d9-f1231ee06193","Type":"ContainerDied","Data":"19a9e3695053a8a9f64745e59bc7ff30dbaed9c9038df39efb2f2a993f9d778f"}
Sep 11 17:56:42.787929 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.787869    2582 scope.go:117] "RemoveContainer" containerID="3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a"
Sep 11 17:56:42.789351 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.789321    2582 generic.go:356] "Generic (PLEG): container finished" podID="51c4b6e4-af79-4e75-a422-60df44dfa18e" containerID="0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7" exitCode=0
Sep 11 17:56:42.789441 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.789368    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"
Sep 11 17:56:42.789483 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.789363    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg" event={"ID":"51c4b6e4-af79-4e75-a422-60df44dfa18e","Type":"ContainerDied","Data":"0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7"}
Sep 11 17:56:42.789546 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.789491    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg" event={"ID":"51c4b6e4-af79-4e75-a422-60df44dfa18e","Type":"ContainerDied","Data":"b8b31e1e617d19ca1eedb874aecb8bc2358b58fcd24aadcdd6ba66f265da1f6c"}
Sep 11 17:56:42.797480 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.797434    2582 scope.go:117] "RemoveContainer" containerID="3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a"
Sep 11 17:56:42.797814 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:56:42.797796    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a\": container with ID starting with 3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a not found: ID does not exist" containerID="3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a"
Sep 11 17:56:42.797867 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.797823    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a"} err="failed to get container status \"3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a\": rpc error: code = NotFound desc = could not find container \"3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a\": container with ID starting with 3551ed28a5a32e081d4bbd153a38d5c5bec9bf013a14b13b9299389971cf6d5a not found: ID does not exist"
Sep 11 17:56:42.797867 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.797846    2582 scope.go:117] "RemoveContainer" containerID="0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7"
Sep 11 17:56:42.806342 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.806162    2582 scope.go:117] "RemoveContainer" containerID="0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7"
Sep 11 17:56:42.806342 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.806285    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"]
Sep 11 17:56:42.806617 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:56:42.806496    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7\": container with ID starting with 0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7 not found: ID does not exist" containerID="0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7"
Sep 11 17:56:42.806617 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.806518    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7"} err="failed to get container status \"0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7\": rpc error: code = NotFound desc = could not find container \"0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7\": container with ID starting with 0d294c9f9705e8105d4945b40acab4b1839c77fbcde1b8c41f308b145939d3d7 not found: ID does not exist"
Sep 11 17:56:42.810798 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.810774    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-104872-56f7db7694-gvvlg"]
Sep 11 17:56:42.822896 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.822868    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"]
Sep 11 17:56:42.829390 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:42.829363    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-104872-6498c7f64b-cxsd6"]
Sep 11 17:56:44.634722 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:44.634636    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f501598-617a-4ff9-a9d9-f1231ee06193" path="/var/lib/kubelet/pods/2f501598-617a-4ff9-a9d9-f1231ee06193/volumes"
Sep 11 17:56:44.635063 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:44.635010    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51c4b6e4-af79-4e75-a422-60df44dfa18e" path="/var/lib/kubelet/pods/51c4b6e4-af79-4e75-a422-60df44dfa18e/volumes"
Sep 11 17:56:45.786151 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786101    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"]
Sep 11 17:56:45.786540 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786524    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abdc631e-8dcd-4baf-a70f-07e96afebea6" containerName="payload-pre-processing"
Sep 11 17:56:45.786580 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786543    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="abdc631e-8dcd-4baf-a70f-07e96afebea6" containerName="payload-pre-processing"
Sep 11 17:56:45.786580 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786566    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f501598-617a-4ff9-a9d9-f1231ee06193" containerName="maas-api"
Sep 11 17:56:45.786580 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786571    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f501598-617a-4ff9-a9d9-f1231ee06193" containerName="maas-api"
Sep 11 17:56:45.786580 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786577    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51c4b6e4-af79-4e75-a422-60df44dfa18e" containerName="payload-pre-processing"
Sep 11 17:56:45.786580 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786582    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="51c4b6e4-af79-4e75-a422-60df44dfa18e" containerName="payload-pre-processing"
Sep 11 17:56:45.786761 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786588    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa" containerName="payload-processing"
Sep 11 17:56:45.786761 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786606    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa" containerName="payload-processing"
Sep 11 17:56:45.786761 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786672    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="51c4b6e4-af79-4e75-a422-60df44dfa18e" containerName="payload-pre-processing"
Sep 11 17:56:45.786761 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786682    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f501598-617a-4ff9-a9d9-f1231ee06193" containerName="maas-api"
Sep 11 17:56:45.786761 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786688    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="53414de3-5bbd-4b32-b2b2-b7a9e7ec58aa" containerName="payload-processing"
Sep 11 17:56:45.786761 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.786694    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="abdc631e-8dcd-4baf-a70f-07e96afebea6" containerName="payload-pre-processing"
Sep 11 17:56:45.791117 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.791075    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:45.794106 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.794075    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"distinct-w0-286c27-kserve-self-signed-certs\""
Sep 11 17:56:45.805357 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.805327    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"]
Sep 11 17:56:45.910533 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.910488    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-model-cache\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:45.910709 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.910554    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-dshm\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:45.910709 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.910595    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tls-certs\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:45.910709 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.910624    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tmp-dir\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:45.910709 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.910683    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq6gd\" (UniqueName: \"kubernetes.io/projected/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-kube-api-access-nq6gd\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:45.910902 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:45.910724    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-home\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.012149 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.012108    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-home\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.012335 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.012194    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-model-cache\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.012335 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.012231    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-dshm\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.012335 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.012266    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tls-certs\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.012335 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.012296    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tmp-dir\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.012335 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.012320    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq6gd\" (UniqueName: \"kubernetes.io/projected/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-kube-api-access-nq6gd\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.012592 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.012499    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-home\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.012592 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.012576    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-model-cache\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.012686 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.012662    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tmp-dir\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.014524 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.014504    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-dshm\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.014825 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.014795    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tls-certs\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.022498 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.022471    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq6gd\" (UniqueName: \"kubernetes.io/projected/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-kube-api-access-nq6gd\") pod \"distinct-w0-286c27-kserve-785c8844f7-m86z8\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.099939 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.099843    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:46.382930 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.382904    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"]
Sep 11 17:56:46.385518 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:56:46.385481    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode144d1b2_48c2_4d3a_97da_99d4c9ee411b.slice/crio-018ef90ca3a8aa5ce0d4e98ea3c88a4e80da295f4902b869d9c6e3062a8ecdc7 WatchSource:0}: Error finding container 018ef90ca3a8aa5ce0d4e98ea3c88a4e80da295f4902b869d9c6e3062a8ecdc7: Status 404 returned error can't find the container with id 018ef90ca3a8aa5ce0d4e98ea3c88a4e80da295f4902b869d9c6e3062a8ecdc7
Sep 11 17:56:46.812449 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.812417    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8" event={"ID":"e144d1b2-48c2-4d3a-97da-99d4c9ee411b","Type":"ContainerStarted","Data":"a3dbfa233da1627d2977d70525b9e3f45d279cb6f8a62bc89585dec677436f1a"}
Sep 11 17:56:46.812846 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.812459    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8" event={"ID":"e144d1b2-48c2-4d3a-97da-99d4c9ee411b","Type":"ContainerStarted","Data":"018ef90ca3a8aa5ce0d4e98ea3c88a4e80da295f4902b869d9c6e3062a8ecdc7"}
Sep 11 17:56:46.846763 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:46.846699    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8" podStartSLOduration=1.846681394 podStartE2EDuration="1.846681394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:56:46.844757498 +0000 UTC m=+1074.732935878" watchObservedRunningTime="2026-09-11 17:56:46.846681394 +0000 UTC m=+1074.734859741"
Sep 11 17:56:49.350686 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:49.350642    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"]
Sep 11 17:56:49.351130 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:49.350867    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy" containerID="cri-o://8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22" gracePeriod=30
Sep 11 17:56:54.612071 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.611999    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:56:54.802409 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802377    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-data\") pod \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") "
Sep 11 17:56:54.802409 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802415    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-podinfo\") pod \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") "
Sep 11 17:56:54.802686 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802450    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-token\") pod \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") "
Sep 11 17:56:54.802686 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802500    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwt88\" (UniqueName: \"kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-kube-api-access-bwt88\") pod \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") "
Sep 11 17:56:54.802686 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802542    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-certs\") pod \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") "
Sep 11 17:56:54.802845 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802697    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae3b765f-5c9d-4cce-9a7d-61846ee013fb" (UID: "ae3b765f-5c9d-4cce-9a7d-61846ee013fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:56:54.802845 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802729    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-credential-socket\") pod \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") "
Sep 11 17:56:54.802845 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802784    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istiod-ca-cert\") pod \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") "
Sep 11 17:56:54.802845 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802819    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae3b765f-5c9d-4cce-9a7d-61846ee013fb" (UID: "ae3b765f-5c9d-4cce-9a7d-61846ee013fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:56:54.802845 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802835    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-socket\") pod \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") "
Sep 11 17:56:54.803103 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802877    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-envoy\") pod \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\" (UID: \"ae3b765f-5c9d-4cce-9a7d-61846ee013fb\") "
Sep 11 17:56:54.803103 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.802948    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae3b765f-5c9d-4cce-9a7d-61846ee013fb" (UID: "ae3b765f-5c9d-4cce-9a7d-61846ee013fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:56:54.803206 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.803117    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae3b765f-5c9d-4cce-9a7d-61846ee013fb" (UID: "ae3b765f-5c9d-4cce-9a7d-61846ee013fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:56:54.803206 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.803129    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae3b765f-5c9d-4cce-9a7d-61846ee013fb" (UID: "ae3b765f-5c9d-4cce-9a7d-61846ee013fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:56:54.803513 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.803489    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-data\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:54.803661 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.803516    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:54.803661 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.803532    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-credential-socket\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:54.803661 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.803546    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istiod-ca-cert\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:54.803661 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.803560    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-workload-socket\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:54.804799 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.804776    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae3b765f-5c9d-4cce-9a7d-61846ee013fb" (UID: "ae3b765f-5c9d-4cce-9a7d-61846ee013fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:56:54.804920 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.804871    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae3b765f-5c9d-4cce-9a7d-61846ee013fb" (UID: "ae3b765f-5c9d-4cce-9a7d-61846ee013fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:56:54.804920 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.804897    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-kube-api-access-bwt88" (OuterVolumeSpecName: "kube-api-access-bwt88") pod "ae3b765f-5c9d-4cce-9a7d-61846ee013fb" (UID: "ae3b765f-5c9d-4cce-9a7d-61846ee013fb"). InnerVolumeSpecName "kube-api-access-bwt88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:56:54.805213 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.805190    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae3b765f-5c9d-4cce-9a7d-61846ee013fb" (UID: "ae3b765f-5c9d-4cce-9a7d-61846ee013fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:56:54.854525 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.854493    2582 generic.go:356] "Generic (PLEG): container finished" podID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerID="8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22" exitCode=0
Sep 11 17:56:54.854709 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.854543    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"
Sep 11 17:56:54.854709 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.854573    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" event={"ID":"ae3b765f-5c9d-4cce-9a7d-61846ee013fb","Type":"ContainerDied","Data":"8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22"}
Sep 11 17:56:54.854709 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.854629    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6" event={"ID":"ae3b765f-5c9d-4cce-9a7d-61846ee013fb","Type":"ContainerDied","Data":"55568cc59db837f5193beac9e1ad60d2dedf92f9ef0a91c7a08a656856c5610a"}
Sep 11 17:56:54.854709 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.854660    2582 scope.go:117] "RemoveContainer" containerID="8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22"
Sep 11 17:56:54.864898 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.864874    2582 scope.go:117] "RemoveContainer" containerID="dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040"
Sep 11 17:56:54.873480 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.873460    2582 scope.go:117] "RemoveContainer" containerID="8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22"
Sep 11 17:56:54.873752 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:56:54.873734    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22\": container with ID starting with 8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22 not found: ID does not exist" containerID="8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22"
Sep 11 17:56:54.873799 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.873762    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22"} err="failed to get container status \"8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22\": rpc error: code = NotFound desc = could not find container \"8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22\": container with ID starting with 8a07004e012fa9871d49da6dc22d0247d19b1944165add8a24ced6162c3edf22 not found: ID does not exist"
Sep 11 17:56:54.873799 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.873779    2582 scope.go:117] "RemoveContainer" containerID="dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040"
Sep 11 17:56:54.874048 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:56:54.874029    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040\": container with ID starting with dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040 not found: ID does not exist" containerID="dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040"
Sep 11 17:56:54.874130 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.874054    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040"} err="failed to get container status \"dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040\": rpc error: code = NotFound desc = could not find container \"dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040\": container with ID starting with dfd72026d3c4d62353910ffb46abd4ad2c89206a7b4536b94b0af016ea6b8040 not found: ID does not exist"
Sep 11 17:56:54.878324 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.878299    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"]
Sep 11 17:56:54.882789 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.882766    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-104872-openshift-default-8d7474ddd-mdqt6"]
Sep 11 17:56:54.904286 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.904258    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-podinfo\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:54.904286 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.904285    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-token\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:54.904473 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.904300    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwt88\" (UniqueName: \"kubernetes.io/projected/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-kube-api-access-bwt88\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:54.904473 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:54.904312    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae3b765f-5c9d-4cce-9a7d-61846ee013fb-istio-envoy\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:56:56.100224 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:56.100188    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:56.100224 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:56.100234    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:56.113049 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:56.113022    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:56.634041 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:56.634006    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" path="/var/lib/kubelet/pods/ae3b765f-5c9d-4cce-9a7d-61846ee013fb/volumes"
Sep 11 17:56:56.877165 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:56.877138    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 17:56:57.242860 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.242827    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"]
Sep 11 17:56:57.243413 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.243395    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy"
Sep 11 17:56:57.243456 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.243416    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy"
Sep 11 17:56:57.243456 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.243429    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy"
Sep 11 17:56:57.243456 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.243438    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy"
Sep 11 17:56:57.243552 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.243536    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy"
Sep 11 17:56:57.243586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.243551    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae3b765f-5c9d-4cce-9a7d-61846ee013fb" containerName="istio-proxy"
Sep 11 17:56:57.248653 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.248631    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.251654 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.251602    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 17:56:57.251792 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.251642    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-21e472-serving-cert\""
Sep 11 17:56:57.251792 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.251685    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 17:56:57.251792 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.251697    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5vrxn\""
Sep 11 17:56:57.251976 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.251798    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:56:57.257994 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.257957    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"]
Sep 11 17:56:57.328307 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.328265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.328514 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.328328    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.328514 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.328426    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcb9b7c-2081-4dad-8dbc-0bec01466729-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.328514 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.328488    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5bw6\" (UniqueName: \"kubernetes.io/projected/2fcb9b7c-2081-4dad-8dbc-0bec01466729-kube-api-access-k5bw6\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.429311 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.429271    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcb9b7c-2081-4dad-8dbc-0bec01466729-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.429500 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.429322    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bw6\" (UniqueName: \"kubernetes.io/projected/2fcb9b7c-2081-4dad-8dbc-0bec01466729-kube-api-access-k5bw6\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.429500 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.429355    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.429500 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.429405    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.429662 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:56:57.429509    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-serving-cert: secret "maas-api-e2e-shared-a-w2-21e472-serving-cert" not found
Sep 11 17:56:57.429662 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:56:57.429577    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-maas-api-tls podName:2fcb9b7c-2081-4dad-8dbc-0bec01466729 nodeName:}" failed. No retries permitted until 2026-09-11 17:56:57.929560349 +0000 UTC m=+1085.817738673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-maas-api-tls") pod "maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj" (UID: "2fcb9b7c-2081-4dad-8dbc-0bec01466729") : secret "maas-api-e2e-shared-a-w2-21e472-serving-cert" not found
Sep 11 17:56:57.429928 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.429909    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcb9b7c-2081-4dad-8dbc-0bec01466729-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.431932 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.431915    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.438099 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.438052    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5bw6\" (UniqueName: \"kubernetes.io/projected/2fcb9b7c-2081-4dad-8dbc-0bec01466729-kube-api-access-k5bw6\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.895845 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.895812    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"]
Sep 11 17:56:57.899678 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.899657    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 17:56:57.902644 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.902619    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-21e472\""
Sep 11 17:56:57.902767 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.902712    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-21e472-dockercfg-tqc2f\""
Sep 11 17:56:57.917629 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.917584    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"]
Sep 11 17:56:57.935474 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.935425    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47rvp\" (UniqueName: \"kubernetes.io/projected/3809c931-66aa-4e3a-80b2-7a25334317db-kube-api-access-47rvp\") pod \"payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz\" (UID: \"3809c931-66aa-4e3a-80b2-7a25334317db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 17:56:57.935666 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.935502    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3809c931-66aa-4e3a-80b2-7a25334317db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz\" (UID: \"3809c931-66aa-4e3a-80b2-7a25334317db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 17:56:57.935666 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.935581    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:57.943169 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:57.943140    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:58.036774 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.036729    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47rvp\" (UniqueName: \"kubernetes.io/projected/3809c931-66aa-4e3a-80b2-7a25334317db-kube-api-access-47rvp\") pod \"payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz\" (UID: \"3809c931-66aa-4e3a-80b2-7a25334317db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 17:56:58.036938 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.036806    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3809c931-66aa-4e3a-80b2-7a25334317db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz\" (UID: \"3809c931-66aa-4e3a-80b2-7a25334317db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 17:56:58.037511 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.037492    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3809c931-66aa-4e3a-80b2-7a25334317db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz\" (UID: \"3809c931-66aa-4e3a-80b2-7a25334317db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 17:56:58.045585 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.045562    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47rvp\" (UniqueName: \"kubernetes.io/projected/3809c931-66aa-4e3a-80b2-7a25334317db-kube-api-access-47rvp\") pod \"payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz\" (UID: \"3809c931-66aa-4e3a-80b2-7a25334317db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 17:56:58.161791 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.161692    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:58.210098 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.210052    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 17:56:58.445696 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.445668    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"]
Sep 11 17:56:58.447550 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:56:58.447526    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fcb9b7c_2081_4dad_8dbc_0bec01466729.slice/crio-7cf711a484cbc5d2632bda6196735e7522168cc8d97e692a1f273baf01038638 WatchSource:0}: Error finding container 7cf711a484cbc5d2632bda6196735e7522168cc8d97e692a1f273baf01038638: Status 404 returned error can't find the container with id 7cf711a484cbc5d2632bda6196735e7522168cc8d97e692a1f273baf01038638
Sep 11 17:56:58.494676 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.494646    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"]
Sep 11 17:56:58.497030 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:56:58.497003    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3809c931_66aa_4e3a_80b2_7a25334317db.slice/crio-1ef54f12855fe02fcb9bed2e842652ad1dd6419d495716e10fa7456f73455412 WatchSource:0}: Error finding container 1ef54f12855fe02fcb9bed2e842652ad1dd6419d495716e10fa7456f73455412: Status 404 returned error can't find the container with id 1ef54f12855fe02fcb9bed2e842652ad1dd6419d495716e10fa7456f73455412
Sep 11 17:56:58.876071 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.876035    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz" event={"ID":"3809c931-66aa-4e3a-80b2-7a25334317db","Type":"ContainerStarted","Data":"401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa"}
Sep 11 17:56:58.876266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.876243    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 17:56:58.876328 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.876289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz" event={"ID":"3809c931-66aa-4e3a-80b2-7a25334317db","Type":"ContainerStarted","Data":"1ef54f12855fe02fcb9bed2e842652ad1dd6419d495716e10fa7456f73455412"}
Sep 11 17:56:58.877376 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.877354    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj" event={"ID":"2fcb9b7c-2081-4dad-8dbc-0bec01466729","Type":"ContainerStarted","Data":"7cf711a484cbc5d2632bda6196735e7522168cc8d97e692a1f273baf01038638"}
Sep 11 17:56:58.894532 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:58.894333    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz" podStartSLOduration=1.894312636 podStartE2EDuration="1.894312636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:56:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:56:58.893711093 +0000 UTC m=+1086.781889438" watchObservedRunningTime="2026-09-11 17:56:58.894312636 +0000 UTC m=+1086.782490984"
Sep 11 17:56:59.883717 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:59.883682    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj" event={"ID":"2fcb9b7c-2081-4dad-8dbc-0bec01466729","Type":"ContainerStarted","Data":"5ff505fc804df7e664cae54254631bd03d6b2744175234cb06c3189b133fa6b0"}
Sep 11 17:56:59.884226 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:59.883966    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:56:59.906129 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:56:59.906044    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj" podStartSLOduration=2.508732138 podStartE2EDuration="2.906020397s" totalImagesPullingTime="397.288259ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:56:59.902125558 +0000 UTC m=+1087.790303905" watchObservedRunningTime="2026-09-11 17:56:59.906020397 +0000 UTC m=+1087.794198746"
Sep 11 17:57:05.893772 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:05.893739    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 17:57:09.886018 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:09.885986    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 17:57:26.170934 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:26.170902    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"]
Sep 11 17:57:33.564752 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.564708    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"]
Sep 11 17:57:33.570596 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.570565    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 17:57:33.573794 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.573769    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-7f85c3-dockercfg-k8f6c\""
Sep 11 17:57:33.573794 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.573772    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-7f85c3\""
Sep 11 17:57:33.578284 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.578254    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"]
Sep 11 17:57:33.588677 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.588646    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"]
Sep 11 17:57:33.592201 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.592178    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 17:57:33.596151 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.596123    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dc40db8-4669-4479-a9ec-c5865f42d394-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759\" (UID: \"3dc40db8-4669-4479-a9ec-c5865f42d394\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 17:57:33.596289 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.596251    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c62v4\" (UniqueName: \"kubernetes.io/projected/3dc40db8-4669-4479-a9ec-c5865f42d394-kube-api-access-c62v4\") pod \"payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759\" (UID: \"3dc40db8-4669-4479-a9ec-c5865f42d394\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 17:57:33.606692 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.606660    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"]
Sep 11 17:57:33.697113 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.697051    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c62v4\" (UniqueName: \"kubernetes.io/projected/3dc40db8-4669-4479-a9ec-c5865f42d394-kube-api-access-c62v4\") pod \"payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759\" (UID: \"3dc40db8-4669-4479-a9ec-c5865f42d394\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 17:57:33.697306 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.697196    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd\" (UID: \"dc0563ad-e78f-46a6-aded-fb4de8f60e3a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 17:57:33.697306 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.697239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dc40db8-4669-4479-a9ec-c5865f42d394-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759\" (UID: \"3dc40db8-4669-4479-a9ec-c5865f42d394\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 17:57:33.697306 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.697273    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdtsg\" (UniqueName: \"kubernetes.io/projected/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-kube-api-access-pdtsg\") pod \"payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd\" (UID: \"dc0563ad-e78f-46a6-aded-fb4de8f60e3a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 17:57:33.697878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.697858    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dc40db8-4669-4479-a9ec-c5865f42d394-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759\" (UID: \"3dc40db8-4669-4479-a9ec-c5865f42d394\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 17:57:33.707459 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.707424    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c62v4\" (UniqueName: \"kubernetes.io/projected/3dc40db8-4669-4479-a9ec-c5865f42d394-kube-api-access-c62v4\") pod \"payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759\" (UID: \"3dc40db8-4669-4479-a9ec-c5865f42d394\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 17:57:33.798731 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.798697    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd\" (UID: \"dc0563ad-e78f-46a6-aded-fb4de8f60e3a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 17:57:33.798904 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.798747    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdtsg\" (UniqueName: \"kubernetes.io/projected/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-kube-api-access-pdtsg\") pod \"payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd\" (UID: \"dc0563ad-e78f-46a6-aded-fb4de8f60e3a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 17:57:33.799412 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.799393    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd\" (UID: \"dc0563ad-e78f-46a6-aded-fb4de8f60e3a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 17:57:33.808140 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.808112    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdtsg\" (UniqueName: \"kubernetes.io/projected/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-kube-api-access-pdtsg\") pod \"payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd\" (UID: \"dc0563ad-e78f-46a6-aded-fb4de8f60e3a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 17:57:33.883197 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.883109    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 17:57:33.904573 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:33.904546    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 17:57:34.165862 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:34.165831    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"]
Sep 11 17:57:34.167976 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:57:34.167948    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc40db8_4669_4479_a9ec_c5865f42d394.slice/crio-5d702dc0e558edf2faad66c6a2b997cb0193644eae457d10c9659f90696cf84f WatchSource:0}: Error finding container 5d702dc0e558edf2faad66c6a2b997cb0193644eae457d10c9659f90696cf84f: Status 404 returned error can't find the container with id 5d702dc0e558edf2faad66c6a2b997cb0193644eae457d10c9659f90696cf84f
Sep 11 17:57:34.190895 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:34.190867    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"]
Sep 11 17:57:34.204344 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:57:34.204299    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0563ad_e78f_46a6_aded_fb4de8f60e3a.slice/crio-50c2d3d0e73933661c4ff766275486579d3e32d7ab8f4b9d3a2dff77d35591b0 WatchSource:0}: Error finding container 50c2d3d0e73933661c4ff766275486579d3e32d7ab8f4b9d3a2dff77d35591b0: Status 404 returned error can't find the container with id 50c2d3d0e73933661c4ff766275486579d3e32d7ab8f4b9d3a2dff77d35591b0
Sep 11 17:57:35.064388 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:35.064350    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd" event={"ID":"dc0563ad-e78f-46a6-aded-fb4de8f60e3a","Type":"ContainerStarted","Data":"3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a"}
Sep 11 17:57:35.064865 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:35.064519    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 17:57:35.064865 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:35.064567    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd" event={"ID":"dc0563ad-e78f-46a6-aded-fb4de8f60e3a","Type":"ContainerStarted","Data":"50c2d3d0e73933661c4ff766275486579d3e32d7ab8f4b9d3a2dff77d35591b0"}
Sep 11 17:57:35.065887 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:35.065866    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759" event={"ID":"3dc40db8-4669-4479-a9ec-c5865f42d394","Type":"ContainerStarted","Data":"6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50"}
Sep 11 17:57:35.065995 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:35.065907    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 17:57:35.065995 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:35.065923    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759" event={"ID":"3dc40db8-4669-4479-a9ec-c5865f42d394","Type":"ContainerStarted","Data":"5d702dc0e558edf2faad66c6a2b997cb0193644eae457d10c9659f90696cf84f"}
Sep 11 17:57:35.082269 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:35.082220    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd" podStartSLOduration=2.082205507 podStartE2EDuration="2.082205507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:57:35.081644282 +0000 UTC m=+1122.969822629" watchObservedRunningTime="2026-09-11 17:57:35.082205507 +0000 UTC m=+1122.970383844"
Sep 11 17:57:35.098530 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:35.098481    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759" podStartSLOduration=2.098466103 podStartE2EDuration="2.098466103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:57:35.097733072 +0000 UTC m=+1122.985911418" watchObservedRunningTime="2026-09-11 17:57:35.098466103 +0000 UTC m=+1122.986644448"
Sep 11 17:57:41.190421 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.190375    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h" podUID="63efa942-3daf-42cf-8667-cb2f88d4c5da" containerName="main" containerID="cri-o://435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040" gracePeriod=45
Sep 11 17:57:41.446571 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.446504    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:57:41.571619 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.571579    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-dshm\") pod \"63efa942-3daf-42cf-8667-cb2f88d4c5da\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") "
Sep 11 17:57:41.571619 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.571624    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63efa942-3daf-42cf-8667-cb2f88d4c5da-tls-certs\") pod \"63efa942-3daf-42cf-8667-cb2f88d4c5da\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") "
Sep 11 17:57:41.571829 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.571660    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44m2g\" (UniqueName: \"kubernetes.io/projected/63efa942-3daf-42cf-8667-cb2f88d4c5da-kube-api-access-44m2g\") pod \"63efa942-3daf-42cf-8667-cb2f88d4c5da\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") "
Sep 11 17:57:41.571829 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.571717    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-model-cache\") pod \"63efa942-3daf-42cf-8667-cb2f88d4c5da\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") "
Sep 11 17:57:41.571829 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.571744    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-home\") pod \"63efa942-3daf-42cf-8667-cb2f88d4c5da\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") "
Sep 11 17:57:41.571956 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.571902    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-tmp-dir\") pod \"63efa942-3daf-42cf-8667-cb2f88d4c5da\" (UID: \"63efa942-3daf-42cf-8667-cb2f88d4c5da\") "
Sep 11 17:57:41.572057 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.572032    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-home" (OuterVolumeSpecName: "home") pod "63efa942-3daf-42cf-8667-cb2f88d4c5da" (UID: "63efa942-3daf-42cf-8667-cb2f88d4c5da"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:57:41.572172 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.572048    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-model-cache" (OuterVolumeSpecName: "model-cache") pod "63efa942-3daf-42cf-8667-cb2f88d4c5da" (UID: "63efa942-3daf-42cf-8667-cb2f88d4c5da"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:57:41.572235 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.572191    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "63efa942-3daf-42cf-8667-cb2f88d4c5da" (UID: "63efa942-3daf-42cf-8667-cb2f88d4c5da"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:57:41.572352 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.572334    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-tmp-dir\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:57:41.572404 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.572356    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-model-cache\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:57:41.572404 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.572370    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-home\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:57:41.574174 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.574150    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63efa942-3daf-42cf-8667-cb2f88d4c5da-kube-api-access-44m2g" (OuterVolumeSpecName: "kube-api-access-44m2g") pod "63efa942-3daf-42cf-8667-cb2f88d4c5da" (UID: "63efa942-3daf-42cf-8667-cb2f88d4c5da"). InnerVolumeSpecName "kube-api-access-44m2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:57:41.574399 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.574383    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-dshm" (OuterVolumeSpecName: "dshm") pod "63efa942-3daf-42cf-8667-cb2f88d4c5da" (UID: "63efa942-3daf-42cf-8667-cb2f88d4c5da"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:57:41.574442 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.574408    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63efa942-3daf-42cf-8667-cb2f88d4c5da-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "63efa942-3daf-42cf-8667-cb2f88d4c5da" (UID: "63efa942-3daf-42cf-8667-cb2f88d4c5da"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:57:41.673132 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.673059    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63efa942-3daf-42cf-8667-cb2f88d4c5da-dshm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:57:41.673132 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.673123    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63efa942-3daf-42cf-8667-cb2f88d4c5da-tls-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:57:41.673132 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:41.673136    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44m2g\" (UniqueName: \"kubernetes.io/projected/63efa942-3daf-42cf-8667-cb2f88d4c5da-kube-api-access-44m2g\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:57:42.108204 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:42.108167    2582 generic.go:356] "Generic (PLEG): container finished" podID="63efa942-3daf-42cf-8667-cb2f88d4c5da" containerID="435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040" exitCode=0
Sep 11 17:57:42.108383 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:42.108218    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"
Sep 11 17:57:42.108383 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:42.108228    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h" event={"ID":"63efa942-3daf-42cf-8667-cb2f88d4c5da","Type":"ContainerDied","Data":"435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040"}
Sep 11 17:57:42.108383 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:42.108266    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h" event={"ID":"63efa942-3daf-42cf-8667-cb2f88d4c5da","Type":"ContainerDied","Data":"81d2625e16c1df0330214c01c73d4d3b6fbc8eda7fcdd1657959ae5ce7316be4"}
Sep 11 17:57:42.108383 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:42.108289    2582 scope.go:117] "RemoveContainer" containerID="435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040"
Sep 11 17:57:42.118426 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:42.118401    2582 scope.go:117] "RemoveContainer" containerID="435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040"
Sep 11 17:57:42.118771 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:57:42.118742    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040\": container with ID starting with 435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040 not found: ID does not exist" containerID="435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040"
Sep 11 17:57:42.118878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:42.118776    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040"} err="failed to get container status \"435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040\": rpc error: code = NotFound desc = could not find container \"435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040\": container with ID starting with 435877fc7f61001162033ea3c4e81cc010d7ce65da63f08e94e6b5fc03592040 not found: ID does not exist"
Sep 11 17:57:42.130669 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:42.130635    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"]
Sep 11 17:57:42.134247 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:42.134225    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-715253/test-model-715253-kserve-6d854f5c79-6gt7h"]
Sep 11 17:57:42.634775 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:42.634740    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63efa942-3daf-42cf-8667-cb2f88d4c5da" path="/var/lib/kubelet/pods/63efa942-3daf-42cf-8667-cb2f88d4c5da/volumes"
Sep 11 17:57:46.073552 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:46.073523    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 17:57:46.074038 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:46.073604    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 17:57:53.667213 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.667173    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8093-2238e3"]
Sep 11 17:57:53.667597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.667570    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63efa942-3daf-42cf-8667-cb2f88d4c5da" containerName="main"
Sep 11 17:57:53.667597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.667580    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="63efa942-3daf-42cf-8667-cb2f88d4c5da" containerName="main"
Sep 11 17:57:53.667669 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.667643    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="63efa942-3daf-42cf-8667-cb2f88d4c5da" containerName="main"
Sep 11 17:57:53.672249 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.672228    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-2238e3"
Sep 11 17:57:53.677720 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.677691    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zsx9b\""
Sep 11 17:57:53.680757 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.680636    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8093-2238e3"]
Sep 11 17:57:53.806760 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.806723    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8t67\" (UniqueName: \"kubernetes.io/projected/ed4b4505-c0be-47ef-bf35-9f99a68a88cb-kube-api-access-t8t67\") pod \"test-curl-8093-2238e3\" (UID: \"ed4b4505-c0be-47ef-bf35-9f99a68a88cb\") " pod="opendatahub/test-curl-8093-2238e3"
Sep 11 17:57:53.907597 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.907558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8t67\" (UniqueName: \"kubernetes.io/projected/ed4b4505-c0be-47ef-bf35-9f99a68a88cb-kube-api-access-t8t67\") pod \"test-curl-8093-2238e3\" (UID: \"ed4b4505-c0be-47ef-bf35-9f99a68a88cb\") " pod="opendatahub/test-curl-8093-2238e3"
Sep 11 17:57:53.918286 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.918215    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8t67\" (UniqueName: \"kubernetes.io/projected/ed4b4505-c0be-47ef-bf35-9f99a68a88cb-kube-api-access-t8t67\") pod \"test-curl-8093-2238e3\" (UID: \"ed4b4505-c0be-47ef-bf35-9f99a68a88cb\") " pod="opendatahub/test-curl-8093-2238e3"
Sep 11 17:57:53.982999 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:53.982965    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-2238e3"
Sep 11 17:57:54.258676 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:54.258521    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8093-2238e3"]
Sep 11 17:57:54.260912 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:57:54.260881    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded4b4505_c0be_47ef_bf35_9f99a68a88cb.slice/crio-b53552c9d41b198d0d22a99eb24af56e044beafb7ae795762ce6d7f0924b9ee3 WatchSource:0}: Error finding container b53552c9d41b198d0d22a99eb24af56e044beafb7ae795762ce6d7f0924b9ee3: Status 404 returned error can't find the container with id b53552c9d41b198d0d22a99eb24af56e044beafb7ae795762ce6d7f0924b9ee3
Sep 11 17:57:55.178242 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:55.178203    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8093-2238e3" event={"ID":"ed4b4505-c0be-47ef-bf35-9f99a68a88cb","Type":"ContainerStarted","Data":"fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156"}
Sep 11 17:57:55.178242 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:55.178246    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8093-2238e3" event={"ID":"ed4b4505-c0be-47ef-bf35-9f99a68a88cb","Type":"ContainerStarted","Data":"b53552c9d41b198d0d22a99eb24af56e044beafb7ae795762ce6d7f0924b9ee3"}
Sep 11 17:57:55.197899 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:55.197848    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8093-2238e3" podStartSLOduration=2.197830697 podStartE2EDuration="2.197830697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:57:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:57:55.195686526 +0000 UTC m=+1143.083864872" watchObservedRunningTime="2026-09-11 17:57:55.197830697 +0000 UTC m=+1143.086009042"
Sep 11 17:57:55.890594 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:55.890549    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8093-2238e3"]
Sep 11 17:57:55.894306 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:55.894280    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8093-2238e3"]
Sep 11 17:57:56.018558 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.018459    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8093-c9e1d6"]
Sep 11 17:57:56.019006 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.018986    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed4b4505-c0be-47ef-bf35-9f99a68a88cb" containerName="test-curl-8093-2238e3"
Sep 11 17:57:56.019113 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.019009    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed4b4505-c0be-47ef-bf35-9f99a68a88cb" containerName="test-curl-8093-2238e3"
Sep 11 17:57:56.019167 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.019134    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed4b4505-c0be-47ef-bf35-9f99a68a88cb" containerName="test-curl-8093-2238e3"
Sep 11 17:57:56.022367 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.022346    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-c9e1d6"
Sep 11 17:57:56.027526 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.027500    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8093-c9e1d6"]
Sep 11 17:57:56.127978 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.127937    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bccq\" (UniqueName: \"kubernetes.io/projected/489cba70-588c-4840-9209-69983b57d222-kube-api-access-9bccq\") pod \"test-curl-8093-c9e1d6\" (UID: \"489cba70-588c-4840-9209-69983b57d222\") " pod="opendatahub/test-curl-8093-c9e1d6"
Sep 11 17:57:56.183339 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:57:56.183308    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8093-2238e3" container="test-curl-8093-2238e3"
Sep 11 17:57:56.228971 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.228934    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bccq\" (UniqueName: \"kubernetes.io/projected/489cba70-588c-4840-9209-69983b57d222-kube-api-access-9bccq\") pod \"test-curl-8093-c9e1d6\" (UID: \"489cba70-588c-4840-9209-69983b57d222\") " pod="opendatahub/test-curl-8093-c9e1d6"
Sep 11 17:57:56.238564 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.238539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bccq\" (UniqueName: \"kubernetes.io/projected/489cba70-588c-4840-9209-69983b57d222-kube-api-access-9bccq\") pod \"test-curl-8093-c9e1d6\" (UID: \"489cba70-588c-4840-9209-69983b57d222\") " pod="opendatahub/test-curl-8093-c9e1d6"
Sep 11 17:57:56.334768 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.334686    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-c9e1d6"
Sep 11 17:57:56.607213 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:56.607130    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8093-c9e1d6"]
Sep 11 17:57:56.611285 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:57:56.611251    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod489cba70_588c_4840_9209_69983b57d222.slice/crio-a4ddcbb4a1b1098291b7b5b8d4c537ff4e14c00f500d9fa757ac0bb8ca712f04 WatchSource:0}: Error finding container a4ddcbb4a1b1098291b7b5b8d4c537ff4e14c00f500d9fa757ac0bb8ca712f04: Status 404 returned error can't find the container with id a4ddcbb4a1b1098291b7b5b8d4c537ff4e14c00f500d9fa757ac0bb8ca712f04
Sep 11 17:57:57.189183 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:57.189148    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8093-c9e1d6" event={"ID":"489cba70-588c-4840-9209-69983b57d222","Type":"ContainerStarted","Data":"0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161"}
Sep 11 17:57:57.189663 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:57.189197    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8093-c9e1d6" event={"ID":"489cba70-588c-4840-9209-69983b57d222","Type":"ContainerStarted","Data":"a4ddcbb4a1b1098291b7b5b8d4c537ff4e14c00f500d9fa757ac0bb8ca712f04"}
Sep 11 17:57:57.189663 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:57.189240    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8093-2238e3" podUID="ed4b4505-c0be-47ef-bf35-9f99a68a88cb" containerName="test-curl-8093-2238e3" containerID="cri-o://fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156" gracePeriod=2
Sep 11 17:57:57.205940 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:57.205895    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8093-c9e1d6" podStartSLOduration=1.205879141 podStartE2EDuration="1.205879141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:57:57.204231656 +0000 UTC m=+1145.092410001" watchObservedRunningTime="2026-09-11 17:57:57.205879141 +0000 UTC m=+1145.094057529"
Sep 11 17:57:57.881062 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:57.881024    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8093-c9e1d6"]
Sep 11 17:57:57.887389 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:57.887362    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8093-c9e1d6"]
Sep 11 17:57:58.027377 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.027294    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8093-871f66"]
Sep 11 17:57:58.027691 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.027678    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="489cba70-588c-4840-9209-69983b57d222" containerName="test-curl-8093-c9e1d6"
Sep 11 17:57:58.027735 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.027692    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="489cba70-588c-4840-9209-69983b57d222" containerName="test-curl-8093-c9e1d6"
Sep 11 17:57:58.027773 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.027761    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="489cba70-588c-4840-9209-69983b57d222" containerName="test-curl-8093-c9e1d6"
Sep 11 17:57:58.030779 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.030762    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-871f66"
Sep 11 17:57:58.043109 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.043042    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8093-871f66"]
Sep 11 17:57:58.150147 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.150069    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxbfd\" (UniqueName: \"kubernetes.io/projected/b825e0fe-6576-4242-9217-fa8cd3bb27f0-kube-api-access-hxbfd\") pod \"test-curl-8093-871f66\" (UID: \"b825e0fe-6576-4242-9217-fa8cd3bb27f0\") " pod="opendatahub/test-curl-8093-871f66"
Sep 11 17:57:58.194689 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:57:58.194657    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8093-c9e1d6" container="test-curl-8093-c9e1d6"
Sep 11 17:57:58.251499 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.251453    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbfd\" (UniqueName: \"kubernetes.io/projected/b825e0fe-6576-4242-9217-fa8cd3bb27f0-kube-api-access-hxbfd\") pod \"test-curl-8093-871f66\" (UID: \"b825e0fe-6576-4242-9217-fa8cd3bb27f0\") " pod="opendatahub/test-curl-8093-871f66"
Sep 11 17:57:58.260512 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.260491    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxbfd\" (UniqueName: \"kubernetes.io/projected/b825e0fe-6576-4242-9217-fa8cd3bb27f0-kube-api-access-hxbfd\") pod \"test-curl-8093-871f66\" (UID: \"b825e0fe-6576-4242-9217-fa8cd3bb27f0\") " pod="opendatahub/test-curl-8093-871f66"
Sep 11 17:57:58.348206 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.348125    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-871f66"
Sep 11 17:57:58.623705 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:58.623680    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8093-871f66"]
Sep 11 17:57:58.625843 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:57:58.625812    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb825e0fe_6576_4242_9217_fa8cd3bb27f0.slice/crio-cb3f2221ea2ffe4f27440277d1ebb92558845830e3c175ccfe841f8d20481096 WatchSource:0}: Error finding container cb3f2221ea2ffe4f27440277d1ebb92558845830e3c175ccfe841f8d20481096: Status 404 returned error can't find the container with id cb3f2221ea2ffe4f27440277d1ebb92558845830e3c175ccfe841f8d20481096
Sep 11 17:57:59.202236 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:59.202204    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8093-871f66" event={"ID":"b825e0fe-6576-4242-9217-fa8cd3bb27f0","Type":"ContainerStarted","Data":"94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8"}
Sep 11 17:57:59.202567 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:59.202252    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8093-871f66" event={"ID":"b825e0fe-6576-4242-9217-fa8cd3bb27f0","Type":"ContainerStarted","Data":"cb3f2221ea2ffe4f27440277d1ebb92558845830e3c175ccfe841f8d20481096"}
Sep 11 17:57:59.202567 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:59.202337    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8093-c9e1d6" podUID="489cba70-588c-4840-9209-69983b57d222" containerName="test-curl-8093-c9e1d6" containerID="cri-o://0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161" gracePeriod=2
Sep 11 17:57:59.224054 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:59.223997    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8093-871f66" podStartSLOduration=1.223978684 podStartE2EDuration="1.223978684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:57:59.223069299 +0000 UTC m=+1147.111247646" watchObservedRunningTime="2026-09-11 17:57:59.223978684 +0000 UTC m=+1147.112157096"
Sep 11 17:57:59.349373 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:59.349346    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-2238e3"
Sep 11 17:57:59.465501 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:59.465406    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8t67\" (UniqueName: \"kubernetes.io/projected/ed4b4505-c0be-47ef-bf35-9f99a68a88cb-kube-api-access-t8t67\") pod \"ed4b4505-c0be-47ef-bf35-9f99a68a88cb\" (UID: \"ed4b4505-c0be-47ef-bf35-9f99a68a88cb\") "
Sep 11 17:57:59.467719 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:59.467687    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed4b4505-c0be-47ef-bf35-9f99a68a88cb-kube-api-access-t8t67" (OuterVolumeSpecName: "kube-api-access-t8t67") pod "ed4b4505-c0be-47ef-bf35-9f99a68a88cb" (UID: "ed4b4505-c0be-47ef-bf35-9f99a68a88cb"). InnerVolumeSpecName "kube-api-access-t8t67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:57:59.567014 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:59.566981    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8t67\" (UniqueName: \"kubernetes.io/projected/ed4b4505-c0be-47ef-bf35-9f99a68a88cb-kube-api-access-t8t67\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:57:59.884393 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:59.884303    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8093-871f66"]
Sep 11 17:57:59.893461 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:57:59.893430    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8093-871f66"]
Sep 11 17:58:00.030824 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.030794    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8093-7f8600"]
Sep 11 17:58:00.031345 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.031329    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b825e0fe-6576-4242-9217-fa8cd3bb27f0" containerName="test-curl-8093-871f66"
Sep 11 17:58:00.031421 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.031347    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b825e0fe-6576-4242-9217-fa8cd3bb27f0" containerName="test-curl-8093-871f66"
Sep 11 17:58:00.031421 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.031417    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b825e0fe-6576-4242-9217-fa8cd3bb27f0" containerName="test-curl-8093-871f66"
Sep 11 17:58:00.034480 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.034459    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-7f8600"
Sep 11 17:58:00.041436 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.041405    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8093-7f8600"]
Sep 11 17:58:00.070975 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.070945    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxf9c\" (UniqueName: \"kubernetes.io/projected/f08e9ac2-abfe-4b5d-81e2-df77553af5a6-kube-api-access-zxf9c\") pod \"test-curl-8093-7f8600\" (UID: \"f08e9ac2-abfe-4b5d-81e2-df77553af5a6\") " pod="opendatahub/test-curl-8093-7f8600"
Sep 11 17:58:00.171654 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.171617    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxf9c\" (UniqueName: \"kubernetes.io/projected/f08e9ac2-abfe-4b5d-81e2-df77553af5a6-kube-api-access-zxf9c\") pod \"test-curl-8093-7f8600\" (UID: \"f08e9ac2-abfe-4b5d-81e2-df77553af5a6\") " pod="opendatahub/test-curl-8093-7f8600"
Sep 11 17:58:00.180914 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.180882    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxf9c\" (UniqueName: \"kubernetes.io/projected/f08e9ac2-abfe-4b5d-81e2-df77553af5a6-kube-api-access-zxf9c\") pod \"test-curl-8093-7f8600\" (UID: \"f08e9ac2-abfe-4b5d-81e2-df77553af5a6\") " pod="opendatahub/test-curl-8093-7f8600"
Sep 11 17:58:00.208463 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.208431    2582 generic.go:356] "Generic (PLEG): container finished" podID="ed4b4505-c0be-47ef-bf35-9f99a68a88cb" containerID="fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156" exitCode=137
Sep 11 17:58:00.208866 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.208469    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-2238e3"
Sep 11 17:58:00.208866 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.208517    2582 scope.go:117] "RemoveContainer" containerID="fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156"
Sep 11 17:58:00.208866 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:00.208747    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8093-871f66" container="test-curl-8093-871f66"
Sep 11 17:58:00.218030 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.218010    2582 scope.go:117] "RemoveContainer" containerID="fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156"
Sep 11 17:58:00.218413 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:00.218394    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156\": container with ID starting with fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156 not found: ID does not exist" containerID="fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156"
Sep 11 17:58:00.218494 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.218427    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156"} err="failed to get container status \"fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156\": rpc error: code = NotFound desc = could not find container \"fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156\": container with ID starting with fa96372f88332f5fc837725992052711d597b9eae8c5e7df198841ee209b5156 not found: ID does not exist"
Sep 11 17:58:00.220196 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:00.220176    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8093-2238e3\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="ed4b4505-c0be-47ef-bf35-9f99a68a88cb" pod="opendatahub/test-curl-8093-2238e3"
Sep 11 17:58:00.346550 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.346517    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-7f8600"
Sep 11 17:58:00.622834 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.622805    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8093-7f8600"]
Sep 11 17:58:00.624775 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:58:00.624738    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf08e9ac2_abfe_4b5d_81e2_df77553af5a6.slice/crio-9949f3a97124bdf1ee2158ba4644749ccf3e49fa88c5d50a9d2c32ece42ff5cf WatchSource:0}: Error finding container 9949f3a97124bdf1ee2158ba4644749ccf3e49fa88c5d50a9d2c32ece42ff5cf: Status 404 returned error can't find the container with id 9949f3a97124bdf1ee2158ba4644749ccf3e49fa88c5d50a9d2c32ece42ff5cf
Sep 11 17:58:00.634809 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:00.634784    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed4b4505-c0be-47ef-bf35-9f99a68a88cb" path="/var/lib/kubelet/pods/ed4b4505-c0be-47ef-bf35-9f99a68a88cb/volumes"
Sep 11 17:58:01.215556 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:01.215525    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8093-7f8600" event={"ID":"f08e9ac2-abfe-4b5d-81e2-df77553af5a6","Type":"ContainerStarted","Data":"63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1"}
Sep 11 17:58:01.215937 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:01.215572    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8093-7f8600" event={"ID":"f08e9ac2-abfe-4b5d-81e2-df77553af5a6","Type":"ContainerStarted","Data":"9949f3a97124bdf1ee2158ba4644749ccf3e49fa88c5d50a9d2c32ece42ff5cf"}
Sep 11 17:58:01.217235 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:01.217204    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8093-871f66" podUID="b825e0fe-6576-4242-9217-fa8cd3bb27f0" containerName="test-curl-8093-871f66" containerID="cri-o://94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8" gracePeriod=2
Sep 11 17:58:01.230204 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:01.230149    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8093-7f8600" podStartSLOduration=1.230131408 podStartE2EDuration="1.230131408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:58:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:58:01.229522298 +0000 UTC m=+1149.117700643" watchObservedRunningTime="2026-09-11 17:58:01.230131408 +0000 UTC m=+1149.118309770"
Sep 11 17:58:01.378495 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:01.378467    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-c9e1d6"
Sep 11 17:58:01.384178 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:01.384153    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bccq\" (UniqueName: \"kubernetes.io/projected/489cba70-588c-4840-9209-69983b57d222-kube-api-access-9bccq\") pod \"489cba70-588c-4840-9209-69983b57d222\" (UID: \"489cba70-588c-4840-9209-69983b57d222\") "
Sep 11 17:58:01.386363 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:01.386331    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/489cba70-588c-4840-9209-69983b57d222-kube-api-access-9bccq" (OuterVolumeSpecName: "kube-api-access-9bccq") pod "489cba70-588c-4840-9209-69983b57d222" (UID: "489cba70-588c-4840-9209-69983b57d222"). InnerVolumeSpecName "kube-api-access-9bccq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:58:01.485610 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:01.485527    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bccq\" (UniqueName: \"kubernetes.io/projected/489cba70-588c-4840-9209-69983b57d222-kube-api-access-9bccq\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:58:01.853490 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:01.853391    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8093-7f8600"]
Sep 11 17:58:01.855499 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:01.855472    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8093-7f8600"]
Sep 11 17:58:02.224015 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:02.223975    2582 generic.go:356] "Generic (PLEG): container finished" podID="489cba70-588c-4840-9209-69983b57d222" containerID="0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161" exitCode=137
Sep 11 17:58:02.224497 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:02.224025    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-c9e1d6"
Sep 11 17:58:02.224497 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:02.224065    2582 scope.go:117] "RemoveContainer" containerID="0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161"
Sep 11 17:58:02.224497 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:02.224479    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8093-7f8600" container="test-curl-8093-7f8600"
Sep 11 17:58:02.235494 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:02.235468    2582 scope.go:117] "RemoveContainer" containerID="0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161"
Sep 11 17:58:02.235566 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:02.235499    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8093-c9e1d6\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="489cba70-588c-4840-9209-69983b57d222" pod="opendatahub/test-curl-8093-c9e1d6"
Sep 11 17:58:02.235987 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:02.235965    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161\": container with ID starting with 0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161 not found: ID does not exist" containerID="0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161"
Sep 11 17:58:02.236060 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:02.235995    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161"} err="failed to get container status \"0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161\": rpc error: code = NotFound desc = could not find container \"0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161\": container with ID starting with 0922416f7a0ae366d1fa587f07c10d4aae2b4e28f8d141d46cce6f363e8f8161 not found: ID does not exist"
Sep 11 17:58:02.632213 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:02.632123    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8093-c9e1d6\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="489cba70-588c-4840-9209-69983b57d222" pod="opendatahub/test-curl-8093-c9e1d6"
Sep 11 17:58:02.634010 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:02.633987    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="489cba70-588c-4840-9209-69983b57d222" path="/var/lib/kubelet/pods/489cba70-588c-4840-9209-69983b57d222/volumes"
Sep 11 17:58:03.230822 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:03.230782    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8093-7f8600" podUID="f08e9ac2-abfe-4b5d-81e2-df77553af5a6" containerName="test-curl-8093-7f8600" containerID="cri-o://63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1" gracePeriod=2
Sep 11 17:58:03.392854 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:03.392828    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-871f66"
Sep 11 17:58:03.406871 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:03.406841    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxbfd\" (UniqueName: \"kubernetes.io/projected/b825e0fe-6576-4242-9217-fa8cd3bb27f0-kube-api-access-hxbfd\") pod \"b825e0fe-6576-4242-9217-fa8cd3bb27f0\" (UID: \"b825e0fe-6576-4242-9217-fa8cd3bb27f0\") "
Sep 11 17:58:03.409041 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:03.409018    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b825e0fe-6576-4242-9217-fa8cd3bb27f0-kube-api-access-hxbfd" (OuterVolumeSpecName: "kube-api-access-hxbfd") pod "b825e0fe-6576-4242-9217-fa8cd3bb27f0" (UID: "b825e0fe-6576-4242-9217-fa8cd3bb27f0"). InnerVolumeSpecName "kube-api-access-hxbfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:58:03.507793 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:03.507705    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxbfd\" (UniqueName: \"kubernetes.io/projected/b825e0fe-6576-4242-9217-fa8cd3bb27f0-kube-api-access-hxbfd\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:58:04.238268 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:04.238233    2582 generic.go:356] "Generic (PLEG): container finished" podID="b825e0fe-6576-4242-9217-fa8cd3bb27f0" containerID="94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8" exitCode=137
Sep 11 17:58:04.238268 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:04.238271    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-871f66"
Sep 11 17:58:04.238748 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:04.238341    2582 scope.go:117] "RemoveContainer" containerID="94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8"
Sep 11 17:58:04.247550 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:04.247527    2582 scope.go:117] "RemoveContainer" containerID="94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8"
Sep 11 17:58:04.247833 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:04.247815    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8\": container with ID starting with 94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8 not found: ID does not exist" containerID="94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8"
Sep 11 17:58:04.247878 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:04.247843    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8"} err="failed to get container status \"94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8\": rpc error: code = NotFound desc = could not find container \"94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8\": container with ID starting with 94a1c2f4f956ed7b1603782cdfdffc79732ff1f548f7bb8221a13456270400b8 not found: ID does not exist"
Sep 11 17:58:04.249260 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:04.249242    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8093-871f66\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="b825e0fe-6576-4242-9217-fa8cd3bb27f0" pod="opendatahub/test-curl-8093-871f66"
Sep 11 17:58:04.635113 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:04.635010    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b825e0fe-6576-4242-9217-fa8cd3bb27f0" path="/var/lib/kubelet/pods/b825e0fe-6576-4242-9217-fa8cd3bb27f0/volumes"
Sep 11 17:58:05.425461 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:05.425433    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-7f8600"
Sep 11 17:58:05.526254 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:05.526172    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxf9c\" (UniqueName: \"kubernetes.io/projected/f08e9ac2-abfe-4b5d-81e2-df77553af5a6-kube-api-access-zxf9c\") pod \"f08e9ac2-abfe-4b5d-81e2-df77553af5a6\" (UID: \"f08e9ac2-abfe-4b5d-81e2-df77553af5a6\") "
Sep 11 17:58:05.528359 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:05.528335    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f08e9ac2-abfe-4b5d-81e2-df77553af5a6-kube-api-access-zxf9c" (OuterVolumeSpecName: "kube-api-access-zxf9c") pod "f08e9ac2-abfe-4b5d-81e2-df77553af5a6" (UID: "f08e9ac2-abfe-4b5d-81e2-df77553af5a6"). InnerVolumeSpecName "kube-api-access-zxf9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:58:05.627547 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:05.627513    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxf9c\" (UniqueName: \"kubernetes.io/projected/f08e9ac2-abfe-4b5d-81e2-df77553af5a6-kube-api-access-zxf9c\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:58:06.251198 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:06.251166    2582 generic.go:356] "Generic (PLEG): container finished" podID="f08e9ac2-abfe-4b5d-81e2-df77553af5a6" containerID="63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1" exitCode=137
Sep 11 17:58:06.251198 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:06.251203    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8093-7f8600"
Sep 11 17:58:06.251395 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:06.251268    2582 scope.go:117] "RemoveContainer" containerID="63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1"
Sep 11 17:58:06.260742 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:06.260721    2582 scope.go:117] "RemoveContainer" containerID="63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1"
Sep 11 17:58:06.261053 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:06.261034    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1\": container with ID starting with 63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1 not found: ID does not exist" containerID="63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1"
Sep 11 17:58:06.261135 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:06.261061    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1"} err="failed to get container status \"63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1\": rpc error: code = NotFound desc = could not find container \"63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1\": container with ID starting with 63344d5c655f7f795dabdf7fbc20e70f2ad99027400348d7d8a5ab1cc9915ec1 not found: ID does not exist"
Sep 11 17:58:06.261929 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:06.261902    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8093-7f8600\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="f08e9ac2-abfe-4b5d-81e2-df77553af5a6" pod="opendatahub/test-curl-8093-7f8600"
Sep 11 17:58:06.636203 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:06.636098    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f08e9ac2-abfe-4b5d-81e2-df77553af5a6" path="/var/lib/kubelet/pods/f08e9ac2-abfe-4b5d-81e2-df77553af5a6/volumes"
Sep 11 17:58:16.333491 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.333399    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"]
Sep 11 17:58:16.333931 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.333658    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr" podUID="bd1bdafc-7907-4eca-8299-265c35f1930f" containerName="payload-processing" containerID="cri-o://976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67" gracePeriod=30
Sep 11 17:58:16.362682 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.362642    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"]
Sep 11 17:58:16.362929 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.362898    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg" podUID="e45c7d90-67ec-490c-bb93-d18c0d6a0616" containerName="payload-pre-processing" containerID="cri-o://58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de" gracePeriod=30
Sep 11 17:58:16.628238 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.628213    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:58:16.640828 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.640807    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:58:16.733094 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.733060    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt4zp\" (UniqueName: \"kubernetes.io/projected/bd1bdafc-7907-4eca-8299-265c35f1930f-kube-api-access-bt4zp\") pod \"bd1bdafc-7907-4eca-8299-265c35f1930f\" (UID: \"bd1bdafc-7907-4eca-8299-265c35f1930f\") "
Sep 11 17:58:16.733274 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.733194    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqrx7\" (UniqueName: \"kubernetes.io/projected/e45c7d90-67ec-490c-bb93-d18c0d6a0616-kube-api-access-wqrx7\") pod \"e45c7d90-67ec-490c-bb93-d18c0d6a0616\" (UID: \"e45c7d90-67ec-490c-bb93-d18c0d6a0616\") "
Sep 11 17:58:16.733274 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.733225    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e45c7d90-67ec-490c-bb93-d18c0d6a0616-plugins-config-volume\") pod \"e45c7d90-67ec-490c-bb93-d18c0d6a0616\" (UID: \"e45c7d90-67ec-490c-bb93-d18c0d6a0616\") "
Sep 11 17:58:16.733274 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.733267    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd1bdafc-7907-4eca-8299-265c35f1930f-plugins-config-volume\") pod \"bd1bdafc-7907-4eca-8299-265c35f1930f\" (UID: \"bd1bdafc-7907-4eca-8299-265c35f1930f\") "
Sep 11 17:58:16.733694 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.733653    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e45c7d90-67ec-490c-bb93-d18c0d6a0616-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e45c7d90-67ec-490c-bb93-d18c0d6a0616" (UID: "e45c7d90-67ec-490c-bb93-d18c0d6a0616"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:58:16.733694 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.733676    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd1bdafc-7907-4eca-8299-265c35f1930f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd1bdafc-7907-4eca-8299-265c35f1930f" (UID: "bd1bdafc-7907-4eca-8299-265c35f1930f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:58:16.733973 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.733953    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e45c7d90-67ec-490c-bb93-d18c0d6a0616-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:58:16.734031 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.733980    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd1bdafc-7907-4eca-8299-265c35f1930f-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:58:16.735643 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.735618    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd1bdafc-7907-4eca-8299-265c35f1930f-kube-api-access-bt4zp" (OuterVolumeSpecName: "kube-api-access-bt4zp") pod "bd1bdafc-7907-4eca-8299-265c35f1930f" (UID: "bd1bdafc-7907-4eca-8299-265c35f1930f"). InnerVolumeSpecName "kube-api-access-bt4zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:58:16.735781 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.735765    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45c7d90-67ec-490c-bb93-d18c0d6a0616-kube-api-access-wqrx7" (OuterVolumeSpecName: "kube-api-access-wqrx7") pod "e45c7d90-67ec-490c-bb93-d18c0d6a0616" (UID: "e45c7d90-67ec-490c-bb93-d18c0d6a0616"). InnerVolumeSpecName "kube-api-access-wqrx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:58:16.834759 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.834726    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqrx7\" (UniqueName: \"kubernetes.io/projected/e45c7d90-67ec-490c-bb93-d18c0d6a0616-kube-api-access-wqrx7\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:58:16.834759 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:16.834754    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt4zp\" (UniqueName: \"kubernetes.io/projected/bd1bdafc-7907-4eca-8299-265c35f1930f-kube-api-access-bt4zp\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 17:58:17.308252 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.308211    2582 generic.go:356] "Generic (PLEG): container finished" podID="e45c7d90-67ec-490c-bb93-d18c0d6a0616" containerID="58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de" exitCode=0
Sep 11 17:58:17.308465 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.308261    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"
Sep 11 17:58:17.308465 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.308303    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg" event={"ID":"e45c7d90-67ec-490c-bb93-d18c0d6a0616","Type":"ContainerDied","Data":"58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de"}
Sep 11 17:58:17.308465 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.308369    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg" event={"ID":"e45c7d90-67ec-490c-bb93-d18c0d6a0616","Type":"ContainerDied","Data":"f8bd780f16dae392a99a15e5a30627295dc332828ee611b77e0cc25c28d21b51"}
Sep 11 17:58:17.308465 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.308398    2582 scope.go:117] "RemoveContainer" containerID="58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de"
Sep 11 17:58:17.309815 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.309792    2582 generic.go:356] "Generic (PLEG): container finished" podID="bd1bdafc-7907-4eca-8299-265c35f1930f" containerID="976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67" exitCode=0
Sep 11 17:58:17.309926 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.309833    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"
Sep 11 17:58:17.309926 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.309905    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr" event={"ID":"bd1bdafc-7907-4eca-8299-265c35f1930f","Type":"ContainerDied","Data":"976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67"}
Sep 11 17:58:17.310039 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.309943    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr" event={"ID":"bd1bdafc-7907-4eca-8299-265c35f1930f","Type":"ContainerDied","Data":"ba6db720ad9792fe5e18237055d8bbc3ea83d298905fe003b1c9862c4c58138e"}
Sep 11 17:58:17.318920 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.318902    2582 scope.go:117] "RemoveContainer" containerID="58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de"
Sep 11 17:58:17.319286 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:17.319259    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de\": container with ID starting with 58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de not found: ID does not exist" containerID="58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de"
Sep 11 17:58:17.319382 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.319294    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de"} err="failed to get container status \"58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de\": rpc error: code = NotFound desc = could not find container \"58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de\": container with ID starting with 58f3a13b8f2502f64132a1b0e9395058629fc197edd3f3df8b45680fa6cbe1de not found: ID does not exist"
Sep 11 17:58:17.319382 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.319318    2582 scope.go:117] "RemoveContainer" containerID="976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67"
Sep 11 17:58:17.327638 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.327621    2582 scope.go:117] "RemoveContainer" containerID="976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67"
Sep 11 17:58:17.327908 ip-10-0-140-215 kubenswrapper[2582]: E0911 17:58:17.327889    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67\": container with ID starting with 976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67 not found: ID does not exist" containerID="976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67"
Sep 11 17:58:17.327961 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.327916    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67"} err="failed to get container status \"976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67\": rpc error: code = NotFound desc = could not find container \"976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67\": container with ID starting with 976df3bb98e137ec13f7305f5e496320f84d29035a872f694a62978489e7bd67 not found: ID does not exist"
Sep 11 17:58:17.337321 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.337295    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"]
Sep 11 17:58:17.352159 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.352137    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9aab53-6dc7975646-lbpfg"]
Sep 11 17:58:17.369465 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.369433    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"]
Sep 11 17:58:17.373020 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:17.372996    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9aab53-6df95f7d4f-jmvsr"]
Sep 11 17:58:18.634241 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:18.634201    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd1bdafc-7907-4eca-8299-265c35f1930f" path="/var/lib/kubelet/pods/bd1bdafc-7907-4eca-8299-265c35f1930f/volumes"
Sep 11 17:58:18.634783 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:18.634762    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e45c7d90-67ec-490c-bb93-d18c0d6a0616" path="/var/lib/kubelet/pods/e45c7d90-67ec-490c-bb93-d18c0d6a0616/volumes"
Sep 11 17:58:21.065392 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.065352    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"]
Sep 11 17:58:21.065898 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.065877    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd1bdafc-7907-4eca-8299-265c35f1930f" containerName="payload-processing"
Sep 11 17:58:21.066000 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.065902    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd1bdafc-7907-4eca-8299-265c35f1930f" containerName="payload-processing"
Sep 11 17:58:21.066000 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.065923    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45c7d90-67ec-490c-bb93-d18c0d6a0616" containerName="payload-pre-processing"
Sep 11 17:58:21.066000 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.065974    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45c7d90-67ec-490c-bb93-d18c0d6a0616" containerName="payload-pre-processing"
Sep 11 17:58:21.066201 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.066051    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f08e9ac2-abfe-4b5d-81e2-df77553af5a6" containerName="test-curl-8093-7f8600"
Sep 11 17:58:21.066201 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.066062    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f08e9ac2-abfe-4b5d-81e2-df77553af5a6" containerName="test-curl-8093-7f8600"
Sep 11 17:58:21.066201 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.066184    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f08e9ac2-abfe-4b5d-81e2-df77553af5a6" containerName="test-curl-8093-7f8600"
Sep 11 17:58:21.066201 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.066199    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd1bdafc-7907-4eca-8299-265c35f1930f" containerName="payload-processing"
Sep 11 17:58:21.066396 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.066213    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e45c7d90-67ec-490c-bb93-d18c0d6a0616" containerName="payload-pre-processing"
Sep 11 17:58:21.069390 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.069367    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.072307 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.072287    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"e2e-conflict-b-2025bfc2-kserve-self-signed-certs\""
Sep 11 17:58:21.078857 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.078832    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"]
Sep 11 17:58:21.177599 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.177539    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-home\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.177599 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.177603    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-dshm\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.177872 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.177655    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-tmp-dir\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.177872 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.177707    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-model-cache\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.177872 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.177792    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a62be2a-8fa9-431b-8640-abcaed03439f-tls-certs\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.177872 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.177858    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj6d2\" (UniqueName: \"kubernetes.io/projected/6a62be2a-8fa9-431b-8640-abcaed03439f-kube-api-access-tj6d2\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.278527 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.278484    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-home\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.278527 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.278544    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-dshm\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.278808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.278584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-tmp-dir\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.278808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.278615    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-model-cache\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.278808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.278664    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a62be2a-8fa9-431b-8640-abcaed03439f-tls-certs\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.278808 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.278701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj6d2\" (UniqueName: \"kubernetes.io/projected/6a62be2a-8fa9-431b-8640-abcaed03439f-kube-api-access-tj6d2\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.279019 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.278955    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-home\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.279019 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.279000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-tmp-dir\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.279172 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.279145    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-model-cache\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.281176 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.281147    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-dshm\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.281339 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.281306    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a62be2a-8fa9-431b-8640-abcaed03439f-tls-certs\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.287923 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.287887    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj6d2\" (UniqueName: \"kubernetes.io/projected/6a62be2a-8fa9-431b-8640-abcaed03439f-kube-api-access-tj6d2\") pod \"e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") " pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.378662 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.378566    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:21.657903 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:21.657880    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"]
Sep 11 17:58:21.659771 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:58:21.659742    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a62be2a_8fa9_431b_8640_abcaed03439f.slice/crio-d4c8d85fc8fb6438fc81b9674d76be3ff59bdc6319f570f5f10f2804d71d0fd7 WatchSource:0}: Error finding container d4c8d85fc8fb6438fc81b9674d76be3ff59bdc6319f570f5f10f2804d71d0fd7: Status 404 returned error can't find the container with id d4c8d85fc8fb6438fc81b9674d76be3ff59bdc6319f570f5f10f2804d71d0fd7
Sep 11 17:58:22.343262 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:22.343225    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc" event={"ID":"6a62be2a-8fa9-431b-8640-abcaed03439f","Type":"ContainerStarted","Data":"81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e"}
Sep 11 17:58:22.343262 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:22.343267    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc" event={"ID":"6a62be2a-8fa9-431b-8640-abcaed03439f","Type":"ContainerStarted","Data":"d4c8d85fc8fb6438fc81b9674d76be3ff59bdc6319f570f5f10f2804d71d0fd7"}
Sep 11 17:58:22.362375 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:22.362329    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc" podStartSLOduration=1.362315033 podStartE2EDuration="1.362315033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:58:22.361271532 +0000 UTC m=+1170.249449878" watchObservedRunningTime="2026-09-11 17:58:22.362315033 +0000 UTC m=+1170.250493429"
Sep 11 17:58:31.379519 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:31.379470    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:31.380058 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:31.379541    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:31.395836 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:31.395805    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:31.407333 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:31.407309    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 17:58:34.074586 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.074550    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"]
Sep 11 17:58:34.078462 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.078446    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 17:58:34.082802 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.082784    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-a18ac6\""
Sep 11 17:58:34.082893 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.082867    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-a18ac6-dockercfg-xlsfl\""
Sep 11 17:58:34.105378 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.105348    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"]
Sep 11 17:58:34.137266 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.137230    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"]
Sep 11 17:58:34.140886 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.140866    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 17:58:34.166852 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.166819    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"]
Sep 11 17:58:34.208154 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.208118    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-692jm\" (UniqueName: \"kubernetes.io/projected/70e673b3-7021-4e59-88ee-3027ee45d1ba-kube-api-access-692jm\") pod \"payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h\" (UID: \"70e673b3-7021-4e59-88ee-3027ee45d1ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 17:58:34.208359 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.208321    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70e673b3-7021-4e59-88ee-3027ee45d1ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h\" (UID: \"70e673b3-7021-4e59-88ee-3027ee45d1ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 17:58:34.309346 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.309303    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tscj\" (UniqueName: \"kubernetes.io/projected/303455fb-866d-40f3-a5c3-4d0e739699af-kube-api-access-2tscj\") pod \"payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4\" (UID: \"303455fb-866d-40f3-a5c3-4d0e739699af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 17:58:34.309564 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.309355    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/303455fb-866d-40f3-a5c3-4d0e739699af-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4\" (UID: \"303455fb-866d-40f3-a5c3-4d0e739699af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 17:58:34.309564 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.309432    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70e673b3-7021-4e59-88ee-3027ee45d1ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h\" (UID: \"70e673b3-7021-4e59-88ee-3027ee45d1ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 17:58:34.309564 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.309492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-692jm\" (UniqueName: \"kubernetes.io/projected/70e673b3-7021-4e59-88ee-3027ee45d1ba-kube-api-access-692jm\") pod \"payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h\" (UID: \"70e673b3-7021-4e59-88ee-3027ee45d1ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 17:58:34.310043 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.310024    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70e673b3-7021-4e59-88ee-3027ee45d1ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h\" (UID: \"70e673b3-7021-4e59-88ee-3027ee45d1ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 17:58:34.319251 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.319223    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-692jm\" (UniqueName: \"kubernetes.io/projected/70e673b3-7021-4e59-88ee-3027ee45d1ba-kube-api-access-692jm\") pod \"payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h\" (UID: \"70e673b3-7021-4e59-88ee-3027ee45d1ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 17:58:34.389885 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.389792    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 17:58:34.410740 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.410704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tscj\" (UniqueName: \"kubernetes.io/projected/303455fb-866d-40f3-a5c3-4d0e739699af-kube-api-access-2tscj\") pod \"payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4\" (UID: \"303455fb-866d-40f3-a5c3-4d0e739699af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 17:58:34.410879 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.410768    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/303455fb-866d-40f3-a5c3-4d0e739699af-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4\" (UID: \"303455fb-866d-40f3-a5c3-4d0e739699af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 17:58:34.411651 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.411624    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/303455fb-866d-40f3-a5c3-4d0e739699af-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4\" (UID: \"303455fb-866d-40f3-a5c3-4d0e739699af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 17:58:34.423834 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.423803    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tscj\" (UniqueName: \"kubernetes.io/projected/303455fb-866d-40f3-a5c3-4d0e739699af-kube-api-access-2tscj\") pod \"payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4\" (UID: \"303455fb-866d-40f3-a5c3-4d0e739699af\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 17:58:34.451034 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.450996    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 17:58:34.673543 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.673514    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"]
Sep 11 17:58:34.675505 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:58:34.675472    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70e673b3_7021_4e59_88ee_3027ee45d1ba.slice/crio-377b9e983f263a21db46102fb5af83177f66b1c2d78be54fc96ac0ec9ce40210 WatchSource:0}: Error finding container 377b9e983f263a21db46102fb5af83177f66b1c2d78be54fc96ac0ec9ce40210: Status 404 returned error can't find the container with id 377b9e983f263a21db46102fb5af83177f66b1c2d78be54fc96ac0ec9ce40210
Sep 11 17:58:34.735768 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:34.735731    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"]
Sep 11 17:58:34.740271 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:58:34.740243    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod303455fb_866d_40f3_a5c3_4d0e739699af.slice/crio-630ea6a85c35a58338f0d0694aae517f6265a00f15c7ebb0613c35826dce603b WatchSource:0}: Error finding container 630ea6a85c35a58338f0d0694aae517f6265a00f15c7ebb0613c35826dce603b: Status 404 returned error can't find the container with id 630ea6a85c35a58338f0d0694aae517f6265a00f15c7ebb0613c35826dce603b
Sep 11 17:58:35.423834 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:35.423790    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4" event={"ID":"303455fb-866d-40f3-a5c3-4d0e739699af","Type":"ContainerStarted","Data":"e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce"}
Sep 11 17:58:35.424332 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:35.423860    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 17:58:35.424332 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:35.423880    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4" event={"ID":"303455fb-866d-40f3-a5c3-4d0e739699af","Type":"ContainerStarted","Data":"630ea6a85c35a58338f0d0694aae517f6265a00f15c7ebb0613c35826dce603b"}
Sep 11 17:58:35.425714 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:35.425686    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h" event={"ID":"70e673b3-7021-4e59-88ee-3027ee45d1ba","Type":"ContainerStarted","Data":"15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e"}
Sep 11 17:58:35.425860 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:35.425740    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h" event={"ID":"70e673b3-7021-4e59-88ee-3027ee45d1ba","Type":"ContainerStarted","Data":"377b9e983f263a21db46102fb5af83177f66b1c2d78be54fc96ac0ec9ce40210"}
Sep 11 17:58:35.425860 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:35.425771    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 17:58:35.446435 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:35.446385    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4" podStartSLOduration=1.446371874 podStartE2EDuration="1.446371874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:58:35.444885381 +0000 UTC m=+1183.333063727" watchObservedRunningTime="2026-09-11 17:58:35.446371874 +0000 UTC m=+1183.334550199"
Sep 11 17:58:35.467640 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:35.467593    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h" podStartSLOduration=1.4675775500000001 podStartE2EDuration="1.46757755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:58:35.464453408 +0000 UTC m=+1183.352631755" watchObservedRunningTime="2026-09-11 17:58:35.46757755 +0000 UTC m=+1183.355755896"
Sep 11 17:58:46.432759 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:46.432728    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 17:58:46.433355 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:58:46.432800    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 17:59:08.763898 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.763862    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"]
Sep 11 17:59:08.769626 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.769602    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 17:59:08.772436 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.772409    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-1bf937\""
Sep 11 17:59:08.780341 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.780313    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"]
Sep 11 17:59:08.783412 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.783384    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"]
Sep 11 17:59:08.787348 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.787329    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 17:59:08.790925 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.790902    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-1bf937-dockercfg-mv7xj\""
Sep 11 17:59:08.792322 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.792010    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"]
Sep 11 17:59:08.848696 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.848662    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab88d592-d281-41d0-b481-50b5c762e493-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n\" (UID: \"ab88d592-d281-41d0-b481-50b5c762e493\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 17:59:08.848865 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.848712    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjshj\" (UniqueName: \"kubernetes.io/projected/ab88d592-d281-41d0-b481-50b5c762e493-kube-api-access-xjshj\") pod \"payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n\" (UID: \"ab88d592-d281-41d0-b481-50b5c762e493\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 17:59:08.950060 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.950025    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjshj\" (UniqueName: \"kubernetes.io/projected/ab88d592-d281-41d0-b481-50b5c762e493-kube-api-access-xjshj\") pod \"payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n\" (UID: \"ab88d592-d281-41d0-b481-50b5c762e493\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 17:59:08.950273 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.950144    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk7f9\" (UniqueName: \"kubernetes.io/projected/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-kube-api-access-lk7f9\") pod \"payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v\" (UID: \"3adfe395-74f6-4d42-b0d0-a6179dcc38eb\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 17:59:08.950273 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.950169    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab88d592-d281-41d0-b481-50b5c762e493-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n\" (UID: \"ab88d592-d281-41d0-b481-50b5c762e493\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 17:59:08.950273 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.950193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v\" (UID: \"3adfe395-74f6-4d42-b0d0-a6179dcc38eb\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 17:59:08.950891 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.950866    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab88d592-d281-41d0-b481-50b5c762e493-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n\" (UID: \"ab88d592-d281-41d0-b481-50b5c762e493\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 17:59:08.958829 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:08.958799    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjshj\" (UniqueName: \"kubernetes.io/projected/ab88d592-d281-41d0-b481-50b5c762e493-kube-api-access-xjshj\") pod \"payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n\" (UID: \"ab88d592-d281-41d0-b481-50b5c762e493\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 17:59:09.050950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.050854    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk7f9\" (UniqueName: \"kubernetes.io/projected/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-kube-api-access-lk7f9\") pod \"payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v\" (UID: \"3adfe395-74f6-4d42-b0d0-a6179dcc38eb\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 17:59:09.050950 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.050906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v\" (UID: \"3adfe395-74f6-4d42-b0d0-a6179dcc38eb\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 17:59:09.051590 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.051572    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v\" (UID: \"3adfe395-74f6-4d42-b0d0-a6179dcc38eb\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 17:59:09.060703 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.060679    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk7f9\" (UniqueName: \"kubernetes.io/projected/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-kube-api-access-lk7f9\") pod \"payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v\" (UID: \"3adfe395-74f6-4d42-b0d0-a6179dcc38eb\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 17:59:09.081677 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.081645    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 17:59:09.099859 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.099834    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 17:59:09.355790 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.355762    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"]
Sep 11 17:59:09.357767 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:59:09.357730    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab88d592_d281_41d0_b481_50b5c762e493.slice/crio-0c72c9b0b449c1b748b250ae08f3942eae5d1136dcc56b694114dd2ced46fdf5 WatchSource:0}: Error finding container 0c72c9b0b449c1b748b250ae08f3942eae5d1136dcc56b694114dd2ced46fdf5: Status 404 returned error can't find the container with id 0c72c9b0b449c1b748b250ae08f3942eae5d1136dcc56b694114dd2ced46fdf5
Sep 11 17:59:09.387043 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.387017    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"]
Sep 11 17:59:09.390308 ip-10-0-140-215 kubenswrapper[2582]: W0911 17:59:09.390280    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3adfe395_74f6_4d42_b0d0_a6179dcc38eb.slice/crio-285a664ab2a8d13afd04a06ef1802bfe57d19e5df95260d73d4121f14f6009c2 WatchSource:0}: Error finding container 285a664ab2a8d13afd04a06ef1802bfe57d19e5df95260d73d4121f14f6009c2: Status 404 returned error can't find the container with id 285a664ab2a8d13afd04a06ef1802bfe57d19e5df95260d73d4121f14f6009c2
Sep 11 17:59:09.605031 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.604921    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n" event={"ID":"ab88d592-d281-41d0-b481-50b5c762e493","Type":"ContainerStarted","Data":"639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae"}
Sep 11 17:59:09.605031 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.604987    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n" event={"ID":"ab88d592-d281-41d0-b481-50b5c762e493","Type":"ContainerStarted","Data":"0c72c9b0b449c1b748b250ae08f3942eae5d1136dcc56b694114dd2ced46fdf5"}
Sep 11 17:59:09.605031 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.605020    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 17:59:09.606577 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.606548    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v" event={"ID":"3adfe395-74f6-4d42-b0d0-a6179dcc38eb","Type":"ContainerStarted","Data":"9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03"}
Sep 11 17:59:09.606718 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.606596    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v" event={"ID":"3adfe395-74f6-4d42-b0d0-a6179dcc38eb","Type":"ContainerStarted","Data":"285a664ab2a8d13afd04a06ef1802bfe57d19e5df95260d73d4121f14f6009c2"}
Sep 11 17:59:09.606718 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.606640    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 17:59:09.622118 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.622051    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n" podStartSLOduration=1.622036258 podStartE2EDuration="1.622036258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:59:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:59:09.620651758 +0000 UTC m=+1217.508830099" watchObservedRunningTime="2026-09-11 17:59:09.622036258 +0000 UTC m=+1217.510215004"
Sep 11 17:59:09.636962 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:09.636907    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v" podStartSLOduration=1.636889015 podStartE2EDuration="1.636889015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:59:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:59:09.63654864 +0000 UTC m=+1217.524726985" watchObservedRunningTime="2026-09-11 17:59:09.636889015 +0000 UTC m=+1217.525067362"
Sep 11 17:59:20.615153 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:20.615115    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 17:59:20.615640 ip-10-0-140-215 kubenswrapper[2582]: I0911 17:59:20.615224    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 18:00:00.162585 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.162552    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th"]
Sep 11 18:00:00.167328 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.167296    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th"
Sep 11 18:00:00.172294 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.171683    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:00:00.176570 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.173522    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k"]
Sep 11 18:00:00.184609 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.184166    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k"
Sep 11 18:00:00.186257 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.186229    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th"
Sep 11 18:00:00.190178 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.190143    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th"]
Sep 11 18:00:00.198343 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.198064    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k"]
Sep 11 18:00:00.199959 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.199925    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl"]
Sep 11 18:00:00.204393 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.204367    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl"
Sep 11 18:00:00.215568 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.215539    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl"]
Sep 11 18:00:00.232687 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.232661    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k"
Sep 11 18:00:00.242815 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.242780    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl"
Sep 11 18:00:00.483032 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.483005    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th"]
Sep 11 18:00:00.485177 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:00:00.485148    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50dc8939_7afd_4a93_b984_304954611f62.slice/crio-b7c2b5613cab4bfe983318836fbeb1eae2647d3097bea19cd2f0e9908ea9efb2 WatchSource:0}: Error finding container b7c2b5613cab4bfe983318836fbeb1eae2647d3097bea19cd2f0e9908ea9efb2: Status 404 returned error can't find the container with id b7c2b5613cab4bfe983318836fbeb1eae2647d3097bea19cd2f0e9908ea9efb2
Sep 11 18:00:00.487051 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.487035    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:00:00.559072 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.559039    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k"]
Sep 11 18:00:00.569242 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:00:00.569189    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d934f1c_22d2_4575_9541_9d6d85965d24.slice/crio-e55433f70bf0b0f3c98a2f64cd4b2136b64af64667f835ef84ec33318854ef9a WatchSource:0}: Error finding container e55433f70bf0b0f3c98a2f64cd4b2136b64af64667f835ef84ec33318854ef9a: Status 404 returned error can't find the container with id e55433f70bf0b0f3c98a2f64cd4b2136b64af64667f835ef84ec33318854ef9a
Sep 11 18:00:00.588444 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.588408    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl"]
Sep 11 18:00:00.592837 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:00:00.592803    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eadc19f_c6dc_4558_88bd_5d0864859e13.slice/crio-ef113e29a2857a42f7704ce0848172cb648d75ef3991b548c705bfe18fec471e WatchSource:0}: Error finding container ef113e29a2857a42f7704ce0848172cb648d75ef3991b548c705bfe18fec471e: Status 404 returned error can't find the container with id ef113e29a2857a42f7704ce0848172cb648d75ef3991b548c705bfe18fec471e
Sep 11 18:00:00.889212 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.889115    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl" event={"ID":"6eadc19f-c6dc-4558-88bd-5d0864859e13","Type":"ContainerStarted","Data":"ef113e29a2857a42f7704ce0848172cb648d75ef3991b548c705bfe18fec471e"}
Sep 11 18:00:00.890695 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.890666    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k" event={"ID":"6d934f1c-22d2-4575-9541-9d6d85965d24","Type":"ContainerStarted","Data":"e55433f70bf0b0f3c98a2f64cd4b2136b64af64667f835ef84ec33318854ef9a"}
Sep 11 18:00:00.892618 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:00.892592    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th" event={"ID":"50dc8939-7afd-4a93-b984-304954611f62","Type":"ContainerStarted","Data":"b7c2b5613cab4bfe983318836fbeb1eae2647d3097bea19cd2f0e9908ea9efb2"}
Sep 11 18:00:01.069245 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:01.069203    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"]
Sep 11 18:00:04.939770 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:04.939727    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl" event={"ID":"6eadc19f-c6dc-4558-88bd-5d0864859e13","Type":"ContainerStarted","Data":"4945dfe2ede720006d9d2691963f4a0b109e5d19e9fdd78f4d2e0b84be2e3a5a"}
Sep 11 18:00:04.942163 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:04.941491    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k" event={"ID":"6d934f1c-22d2-4575-9541-9d6d85965d24","Type":"ContainerStarted","Data":"fa7951f2812d66f995d4d828dbfb55b00f253a54ee10e94a1612c996da90f3e4"}
Sep 11 18:00:04.943101 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:04.943054    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th" event={"ID":"50dc8939-7afd-4a93-b984-304954611f62","Type":"ContainerStarted","Data":"9725acbd1c6e18230ca95e3c9b9f0fab18c1ea2ead9529dff174efa15f5a3856"}
Sep 11 18:00:04.960286 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:04.960230    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl" podStartSLOduration=1.5295410170000001 podStartE2EDuration="4.960208727s" totalImagesPullingTime="3.43066771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:04.959258752 +0000 UTC m=+1272.847437098" watchObservedRunningTime="2026-09-11 18:00:04.960208727 +0000 UTC m=+1272.848387074"
Sep 11 18:00:04.979679 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:04.979628    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th" podStartSLOduration=1.441232334 podStartE2EDuration="4.979612553s" totalImagesPullingTime="3.538380219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:04.977672714 +0000 UTC m=+1272.865851055" watchObservedRunningTime="2026-09-11 18:00:04.979612553 +0000 UTC m=+1272.867790899"
Sep 11 18:00:05.001389 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:05.001325    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k" podStartSLOduration=1.5432388719999999 podStartE2EDuration="5.001309349s" totalImagesPullingTime="3.458070477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:04.999345016 +0000 UTC m=+1272.887523362" watchObservedRunningTime="2026-09-11 18:00:05.001309349 +0000 UTC m=+1272.889487694"
Sep 11 18:00:05.178366 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:00:05.178326    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eadc19f_c6dc_4558_88bd_5d0864859e13.slice/crio-conmon-4945dfe2ede720006d9d2691963f4a0b109e5d19e9fdd78f4d2e0b84be2e3a5a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50dc8939_7afd_4a93_b984_304954611f62.slice/crio-conmon-9725acbd1c6e18230ca95e3c9b9f0fab18c1ea2ead9529dff174efa15f5a3856.scope\": RecentStats: unable to find data in memory cache]"
Sep 11 18:00:05.949108 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:05.949059    2582 generic.go:356] "Generic (PLEG): container finished" podID="6eadc19f-c6dc-4558-88bd-5d0864859e13" containerID="4945dfe2ede720006d9d2691963f4a0b109e5d19e9fdd78f4d2e0b84be2e3a5a" exitCode=0
Sep 11 18:00:05.949564 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:05.949117    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl" event={"ID":"6eadc19f-c6dc-4558-88bd-5d0864859e13","Type":"ContainerDied","Data":"4945dfe2ede720006d9d2691963f4a0b109e5d19e9fdd78f4d2e0b84be2e3a5a"}
Sep 11 18:00:05.950757 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:05.950737    2582 generic.go:356] "Generic (PLEG): container finished" podID="6d934f1c-22d2-4575-9541-9d6d85965d24" containerID="fa7951f2812d66f995d4d828dbfb55b00f253a54ee10e94a1612c996da90f3e4" exitCode=0
Sep 11 18:00:05.950877 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:05.950809    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k" event={"ID":"6d934f1c-22d2-4575-9541-9d6d85965d24","Type":"ContainerDied","Data":"fa7951f2812d66f995d4d828dbfb55b00f253a54ee10e94a1612c996da90f3e4"}
Sep 11 18:00:05.952205 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:05.952185    2582 generic.go:356] "Generic (PLEG): container finished" podID="50dc8939-7afd-4a93-b984-304954611f62" containerID="9725acbd1c6e18230ca95e3c9b9f0fab18c1ea2ead9529dff174efa15f5a3856" exitCode=0
Sep 11 18:00:05.952299 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:05.952264    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th" event={"ID":"50dc8939-7afd-4a93-b984-304954611f62","Type":"ContainerDied","Data":"9725acbd1c6e18230ca95e3c9b9f0fab18c1ea2ead9529dff174efa15f5a3856"}
Sep 11 18:00:07.103171 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.103146    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k"
Sep 11 18:00:07.160242 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.160212    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl"
Sep 11 18:00:07.178981 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.178954    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th"
Sep 11 18:00:07.964757 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.964724    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl"
Sep 11 18:00:07.964942 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.964755    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl" event={"ID":"6eadc19f-c6dc-4558-88bd-5d0864859e13","Type":"ContainerDied","Data":"ef113e29a2857a42f7704ce0848172cb648d75ef3991b548c705bfe18fec471e"}
Sep 11 18:00:07.964942 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.964806    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef113e29a2857a42f7704ce0848172cb648d75ef3991b548c705bfe18fec471e"
Sep 11 18:00:07.966153 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.966129    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k"
Sep 11 18:00:07.966153 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.966133    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k" event={"ID":"6d934f1c-22d2-4575-9541-9d6d85965d24","Type":"ContainerDied","Data":"e55433f70bf0b0f3c98a2f64cd4b2136b64af64667f835ef84ec33318854ef9a"}
Sep 11 18:00:07.966349 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.966178    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e55433f70bf0b0f3c98a2f64cd4b2136b64af64667f835ef84ec33318854ef9a"
Sep 11 18:00:07.967410 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.967389    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th"
Sep 11 18:00:07.967519 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.967391    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th" event={"ID":"50dc8939-7afd-4a93-b984-304954611f62","Type":"ContainerDied","Data":"b7c2b5613cab4bfe983318836fbeb1eae2647d3097bea19cd2f0e9908ea9efb2"}
Sep 11 18:00:07.967562 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:07.967526    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7c2b5613cab4bfe983318836fbeb1eae2647d3097bea19cd2f0e9908ea9efb2"
Sep 11 18:00:16.094169 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.094121    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc" podUID="6a62be2a-8fa9-431b-8640-abcaed03439f" containerName="main" containerID="cri-o://81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e" gracePeriod=45
Sep 11 18:00:16.355562 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.355489    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 18:00:16.422907 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.422872    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj6d2\" (UniqueName: \"kubernetes.io/projected/6a62be2a-8fa9-431b-8640-abcaed03439f-kube-api-access-tj6d2\") pod \"6a62be2a-8fa9-431b-8640-abcaed03439f\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") "
Sep 11 18:00:16.423096 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.422917    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-tmp-dir\") pod \"6a62be2a-8fa9-431b-8640-abcaed03439f\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") "
Sep 11 18:00:16.423096 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.422950    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-model-cache\") pod \"6a62be2a-8fa9-431b-8640-abcaed03439f\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") "
Sep 11 18:00:16.423096 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.423028    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-home\") pod \"6a62be2a-8fa9-431b-8640-abcaed03439f\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") "
Sep 11 18:00:16.423096 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.423053    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a62be2a-8fa9-431b-8640-abcaed03439f-tls-certs\") pod \"6a62be2a-8fa9-431b-8640-abcaed03439f\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") "
Sep 11 18:00:16.423307 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.423164    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-dshm\") pod \"6a62be2a-8fa9-431b-8640-abcaed03439f\" (UID: \"6a62be2a-8fa9-431b-8640-abcaed03439f\") "
Sep 11 18:00:16.423307 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.423224    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-model-cache" (OuterVolumeSpecName: "model-cache") pod "6a62be2a-8fa9-431b-8640-abcaed03439f" (UID: "6a62be2a-8fa9-431b-8640-abcaed03439f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:00:16.423307 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.423253    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6a62be2a-8fa9-431b-8640-abcaed03439f" (UID: "6a62be2a-8fa9-431b-8640-abcaed03439f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:00:16.423471 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.423300    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-home" (OuterVolumeSpecName: "home") pod "6a62be2a-8fa9-431b-8640-abcaed03439f" (UID: "6a62be2a-8fa9-431b-8640-abcaed03439f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:00:16.423532 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.423514    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-tmp-dir\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:16.423532 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.423530    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-model-cache\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:16.423611 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.423546    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-home\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:16.425559 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.425526    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-dshm" (OuterVolumeSpecName: "dshm") pod "6a62be2a-8fa9-431b-8640-abcaed03439f" (UID: "6a62be2a-8fa9-431b-8640-abcaed03439f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:00:16.425559 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.425532    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a62be2a-8fa9-431b-8640-abcaed03439f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6a62be2a-8fa9-431b-8640-abcaed03439f" (UID: "6a62be2a-8fa9-431b-8640-abcaed03439f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:00:16.425708 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.425616    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a62be2a-8fa9-431b-8640-abcaed03439f-kube-api-access-tj6d2" (OuterVolumeSpecName: "kube-api-access-tj6d2") pod "6a62be2a-8fa9-431b-8640-abcaed03439f" (UID: "6a62be2a-8fa9-431b-8640-abcaed03439f"). InnerVolumeSpecName "kube-api-access-tj6d2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:00:16.524903 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.524865    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj6d2\" (UniqueName: \"kubernetes.io/projected/6a62be2a-8fa9-431b-8640-abcaed03439f-kube-api-access-tj6d2\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:16.524903 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.524898    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a62be2a-8fa9-431b-8640-abcaed03439f-tls-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:16.524903 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:16.524908    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a62be2a-8fa9-431b-8640-abcaed03439f-dshm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:17.015760 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:17.015718    2582 generic.go:356] "Generic (PLEG): container finished" podID="6a62be2a-8fa9-431b-8640-abcaed03439f" containerID="81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e" exitCode=0
Sep 11 18:00:17.015935 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:17.015828    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"
Sep 11 18:00:17.015935 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:17.015856    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc" event={"ID":"6a62be2a-8fa9-431b-8640-abcaed03439f","Type":"ContainerDied","Data":"81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e"}
Sep 11 18:00:17.015935 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:17.015910    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc" event={"ID":"6a62be2a-8fa9-431b-8640-abcaed03439f","Type":"ContainerDied","Data":"d4c8d85fc8fb6438fc81b9674d76be3ff59bdc6319f570f5f10f2804d71d0fd7"}
Sep 11 18:00:17.016682 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:17.015929    2582 scope.go:117] "RemoveContainer" containerID="81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e"
Sep 11 18:00:17.026245 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:17.026228    2582 scope.go:117] "RemoveContainer" containerID="81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e"
Sep 11 18:00:17.026549 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:00:17.026531    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e\": container with ID starting with 81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e not found: ID does not exist" containerID="81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e"
Sep 11 18:00:17.026604 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:17.026558    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e"} err="failed to get container status \"81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e\": rpc error: code = NotFound desc = could not find container \"81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e\": container with ID starting with 81558ad63aac657a3acd3e035e496c5fac5c8040de8eaab1b4e07bd7fda4515e not found: ID does not exist"
Sep 11 18:00:17.046786 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:17.046757    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"]
Sep 11 18:00:17.051214 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:17.051190    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-b-2025bfc2-kserve-578876c5bf-l86dc"]
Sep 11 18:00:18.633804 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:18.633768    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a62be2a-8fa9-431b-8640-abcaed03439f" path="/var/lib/kubelet/pods/6a62be2a-8fa9-431b-8640-abcaed03439f/volumes"
Sep 11 18:00:20.470146 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:20.470114    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"]
Sep 11 18:00:35.489365 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.489294    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4" podUID="8301717f-ce6a-4edc-b17d-208645010ef0" containerName="main" containerID="cri-o://39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855" gracePeriod=45
Sep 11 18:00:35.756899 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.756873    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 18:00:35.830367 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830331    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-model-cache\") pod \"8301717f-ce6a-4edc-b17d-208645010ef0\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") "
Sep 11 18:00:35.830367 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830386    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-dshm\") pod \"8301717f-ce6a-4edc-b17d-208645010ef0\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") "
Sep 11 18:00:35.830634 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830411    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-tmp-dir\") pod \"8301717f-ce6a-4edc-b17d-208645010ef0\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") "
Sep 11 18:00:35.830634 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830431    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-home\") pod \"8301717f-ce6a-4edc-b17d-208645010ef0\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") "
Sep 11 18:00:35.830634 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830468    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbkkw\" (UniqueName: \"kubernetes.io/projected/8301717f-ce6a-4edc-b17d-208645010ef0-kube-api-access-pbkkw\") pod \"8301717f-ce6a-4edc-b17d-208645010ef0\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") "
Sep 11 18:00:35.830634 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830511    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8301717f-ce6a-4edc-b17d-208645010ef0-tls-certs\") pod \"8301717f-ce6a-4edc-b17d-208645010ef0\" (UID: \"8301717f-ce6a-4edc-b17d-208645010ef0\") "
Sep 11 18:00:35.830851 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830678    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8301717f-ce6a-4edc-b17d-208645010ef0" (UID: "8301717f-ce6a-4edc-b17d-208645010ef0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:00:35.830851 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830713    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-model-cache" (OuterVolumeSpecName: "model-cache") pod "8301717f-ce6a-4edc-b17d-208645010ef0" (UID: "8301717f-ce6a-4edc-b17d-208645010ef0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:00:35.830851 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830753    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-home" (OuterVolumeSpecName: "home") pod "8301717f-ce6a-4edc-b17d-208645010ef0" (UID: "8301717f-ce6a-4edc-b17d-208645010ef0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:00:35.830948 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830854    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-model-cache\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:35.830948 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830873    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-tmp-dir\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:35.830948 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.830885    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-home\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:35.832848 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.832827    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-dshm" (OuterVolumeSpecName: "dshm") pod "8301717f-ce6a-4edc-b17d-208645010ef0" (UID: "8301717f-ce6a-4edc-b17d-208645010ef0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:00:35.832848 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.832836    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8301717f-ce6a-4edc-b17d-208645010ef0-kube-api-access-pbkkw" (OuterVolumeSpecName: "kube-api-access-pbkkw") pod "8301717f-ce6a-4edc-b17d-208645010ef0" (UID: "8301717f-ce6a-4edc-b17d-208645010ef0"). InnerVolumeSpecName "kube-api-access-pbkkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:00:35.832984 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.832919    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8301717f-ce6a-4edc-b17d-208645010ef0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8301717f-ce6a-4edc-b17d-208645010ef0" (UID: "8301717f-ce6a-4edc-b17d-208645010ef0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:00:35.931651 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.931614    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8301717f-ce6a-4edc-b17d-208645010ef0-dshm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:35.931651 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.931649    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbkkw\" (UniqueName: \"kubernetes.io/projected/8301717f-ce6a-4edc-b17d-208645010ef0-kube-api-access-pbkkw\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:35.931864 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:35.931665    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8301717f-ce6a-4edc-b17d-208645010ef0-tls-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:00:36.120894 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:36.120796    2582 generic.go:356] "Generic (PLEG): container finished" podID="8301717f-ce6a-4edc-b17d-208645010ef0" containerID="39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855" exitCode=0
Sep 11 18:00:36.120894 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:36.120842    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"
Sep 11 18:00:36.120894 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:36.120853    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4" event={"ID":"8301717f-ce6a-4edc-b17d-208645010ef0","Type":"ContainerDied","Data":"39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855"}
Sep 11 18:00:36.120894 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:36.120893    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4" event={"ID":"8301717f-ce6a-4edc-b17d-208645010ef0","Type":"ContainerDied","Data":"1db402d9aaac40193eb7cd7d9a43fda2d0b3b078c510dc6c0c6a420d716b8b24"}
Sep 11 18:00:36.121215 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:36.120914    2582 scope.go:117] "RemoveContainer" containerID="39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855"
Sep 11 18:00:36.131714 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:36.131693    2582 scope.go:117] "RemoveContainer" containerID="39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855"
Sep 11 18:00:36.131983 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:00:36.131962    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855\": container with ID starting with 39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855 not found: ID does not exist" containerID="39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855"
Sep 11 18:00:36.132047 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:36.131994    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855"} err="failed to get container status \"39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855\": rpc error: code = NotFound desc = could not find container \"39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855\": container with ID starting with 39b6698d815217c92d22b0fec6723fec147375dc551511c76497a5546d8ac855 not found: ID does not exist"
Sep 11 18:00:36.144162 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:36.144133    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"]
Sep 11 18:00:36.150164 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:36.150138    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/e2e-conflict-a-2025bfc2-kserve-77cdb5bcf-hl5f4"]
Sep 11 18:00:36.634058 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:36.634025    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8301717f-ce6a-4edc-b17d-208645010ef0" path="/var/lib/kubelet/pods/8301717f-ce6a-4edc-b17d-208645010ef0/volumes"
Sep 11 18:00:48.316146 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316109    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"]
Sep 11 18:00:48.316582 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316568    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8301717f-ce6a-4edc-b17d-208645010ef0" containerName="main"
Sep 11 18:00:48.316627 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316584    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8301717f-ce6a-4edc-b17d-208645010ef0" containerName="main"
Sep 11 18:00:48.316627 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316600    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a62be2a-8fa9-431b-8640-abcaed03439f" containerName="main"
Sep 11 18:00:48.316627 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316606    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a62be2a-8fa9-431b-8640-abcaed03439f" containerName="main"
Sep 11 18:00:48.316627 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316612    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d934f1c-22d2-4575-9541-9d6d85965d24" containerName="cleanup"
Sep 11 18:00:48.316627 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316618    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d934f1c-22d2-4575-9541-9d6d85965d24" containerName="cleanup"
Sep 11 18:00:48.316787 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316627    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50dc8939-7afd-4a93-b984-304954611f62" containerName="cleanup"
Sep 11 18:00:48.316787 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316636    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="50dc8939-7afd-4a93-b984-304954611f62" containerName="cleanup"
Sep 11 18:00:48.316787 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316652    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6eadc19f-c6dc-4558-88bd-5d0864859e13" containerName="cleanup"
Sep 11 18:00:48.316787 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316657    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6eadc19f-c6dc-4558-88bd-5d0864859e13" containerName="cleanup"
Sep 11 18:00:48.316787 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316729    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8301717f-ce6a-4edc-b17d-208645010ef0" containerName="main"
Sep 11 18:00:48.316787 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316737    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="50dc8939-7afd-4a93-b984-304954611f62" containerName="cleanup"
Sep 11 18:00:48.316787 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316744    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a62be2a-8fa9-431b-8640-abcaed03439f" containerName="main"
Sep 11 18:00:48.316787 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316752    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d934f1c-22d2-4575-9541-9d6d85965d24" containerName="cleanup"
Sep 11 18:00:48.316787 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.316757    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6eadc19f-c6dc-4558-88bd-5d0864859e13" containerName="cleanup"
Sep 11 18:00:48.321553 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.321522    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.326239 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.326213    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"kube-root-ca.crt\""
Sep 11 18:00:48.326239 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.326232    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"distinct-w4-4ab253-kserve-self-signed-certs\""
Sep 11 18:00:48.326452 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.326266    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"default-dockercfg-8n74d\""
Sep 11 18:00:48.326452 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.326291    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"openshift-service-ca.crt\""
Sep 11 18:00:48.339433 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.339405    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"]
Sep 11 18:00:48.349666 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.349642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-dshm\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.349807 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.349677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p2wk\" (UniqueName: \"kubernetes.io/projected/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-kube-api-access-6p2wk\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.349807 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.349698    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tls-certs\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.349807 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.349797    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-model-cache\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.349915 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.349843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tmp-dir\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.349955 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.349921    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-home\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.450764 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.450716    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p2wk\" (UniqueName: \"kubernetes.io/projected/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-kube-api-access-6p2wk\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.450764 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.450769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tls-certs\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.451001 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.450816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-model-cache\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.451001 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.450842    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tmp-dir\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.451001 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.450919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-home\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.451001 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.450962    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-dshm\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.451332 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.451310    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-model-cache\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.451376 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.451324    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-home\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.451411 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.451366    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tmp-dir\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.453346 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.453327    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-dshm\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.453492 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.453476    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tls-certs\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.465436 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.465412    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p2wk\" (UniqueName: \"kubernetes.io/projected/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-kube-api-access-6p2wk\") pod \"distinct-w4-4ab253-kserve-54dbc46cc6-bv85b\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") " pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.631359 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.631282    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:48.910109 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:48.905399    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"]
Sep 11 18:00:49.202202 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:49.202068    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b" event={"ID":"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57","Type":"ContainerStarted","Data":"17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf"}
Sep 11 18:00:49.202202 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:49.202142    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b" event={"ID":"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57","Type":"ContainerStarted","Data":"0c66977cb92080cf3eceee763a9b3d222bef92c8ff92f794a82e9eda5fafc109"}
Sep 11 18:00:49.222924 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:49.222874    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b" podStartSLOduration=1.222857883 podStartE2EDuration="1.222857883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:49.220876301 +0000 UTC m=+1317.109054646" watchObservedRunningTime="2026-09-11 18:00:49.222857883 +0000 UTC m=+1317.111036230"
Sep 11 18:00:58.641072 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:58.641039    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:58.641491 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:58.641103    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:58.650434 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:58.650408    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:00:59.269059 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:00:59.269027    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:01:50.368532 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.368490    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"]
Sep 11 18:01:50.371227 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.371205    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.374270 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.374244    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"embedding-w0-286c27-kserve-self-signed-certs\""
Sep 11 18:01:50.385796 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.385765    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"]
Sep 11 18:01:50.471574 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.471534    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-dshm\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.471773 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.471583    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tls-certs\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.471773 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.471669    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf7ms\" (UniqueName: \"kubernetes.io/projected/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-kube-api-access-lf7ms\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.471773 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.471699    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tmp-dir\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.471773 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.471772    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-model-cache\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.471990 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.471805    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-home\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.572448 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.572405    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf7ms\" (UniqueName: \"kubernetes.io/projected/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-kube-api-access-lf7ms\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.572448 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.572450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tmp-dir\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.572701 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.572492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-model-cache\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.572701 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.572524    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-home\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.572701 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.572552    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-dshm\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.572701 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.572587    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tls-certs\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.572981 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.572955    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-model-cache\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.573057 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.572966    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tmp-dir\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.573057 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.573010    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-home\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.575201 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.575177    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-dshm\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.575348 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.575332    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tls-certs\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.580879 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.580856    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf7ms\" (UniqueName: \"kubernetes.io/projected/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-kube-api-access-lf7ms\") pod \"embedding-w0-286c27-kserve-5fc655b746-zrbkk\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") " pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.681052 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.681016    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:01:50.956865 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:50.956841    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"]
Sep 11 18:01:50.959048 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:01:50.959022    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ca43de3_32ec_4ad1_b0ba_4e0b695f3770.slice/crio-7a1984a86259c49df9a0c254b88525b3176fb13ca5265e491f78312af936b613 WatchSource:0}: Error finding container 7a1984a86259c49df9a0c254b88525b3176fb13ca5265e491f78312af936b613: Status 404 returned error can't find the container with id 7a1984a86259c49df9a0c254b88525b3176fb13ca5265e491f78312af936b613
Sep 11 18:01:51.531014 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:51.530979    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk" event={"ID":"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770","Type":"ContainerStarted","Data":"7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4"}
Sep 11 18:01:51.531434 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:51.531026    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk" event={"ID":"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770","Type":"ContainerStarted","Data":"7a1984a86259c49df9a0c254b88525b3176fb13ca5265e491f78312af936b613"}
Sep 11 18:01:51.551427 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:01:51.551369    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk" podStartSLOduration=1.551352347 podStartE2EDuration="1.551352347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:01:51.549330601 +0000 UTC m=+1379.437508946" watchObservedRunningTime="2026-09-11 18:01:51.551352347 +0000 UTC m=+1379.439530692"
Sep 11 18:02:00.681465 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:02:00.681429    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:02:00.681858 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:02:00.681483    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:02:00.694344 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:02:00.694317    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:02:01.594629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:02:01.594590    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:03:14.711406 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:14.711362    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"]
Sep 11 18:03:14.718881 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:14.718848    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"]
Sep 11 18:03:14.734781 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:14.734747    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"]
Sep 11 18:03:14.741353 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:14.741325    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"]
Sep 11 18:03:29.732303 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.732243    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8" podUID="e144d1b2-48c2-4d3a-97da-99d4c9ee411b" containerName="main" containerID="cri-o://a3dbfa233da1627d2977d70525b9e3f45d279cb6f8a62bc89585dec677436f1a" gracePeriod=45
Sep 11 18:03:29.743552 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.743483    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk" podUID="4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" containerName="main" containerID="cri-o://7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4" gracePeriod=45
Sep 11 18:03:29.754730 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.754687    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw" podUID="42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" containerName="main" containerID="cri-o://4c8c6cc6fdcf245bed453c1833d5a655cf1c54509013ab4629cc8605b9fcc590" gracePeriod=45
Sep 11 18:03:29.757945 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.757918    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"]
Sep 11 18:03:29.760729 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.760702    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm" podUID="78f62f94-9e42-4bca-bd2c-898cd8d1c807" containerName="main" containerID="cri-o://0f7bff50d2ed0356d83571c0ef05aef25b3af2845cc9c97e387a82a6f2b52731" gracePeriod=45
Sep 11 18:03:29.764604 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.764586    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:03:29.769075 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.769043    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-a93550\""
Sep 11 18:03:29.771509 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.771485    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"]
Sep 11 18:03:29.806877 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.806846    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"]
Sep 11 18:03:29.810144 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.810128    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:03:29.814165 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.814144    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-a93550-dockercfg-54gg8\""
Sep 11 18:03:29.823260 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.823232    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"]
Sep 11 18:03:29.866660 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.866618    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9812ee9c-19b5-461a-b579-ad6b0b80bd83-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4\" (UID: \"9812ee9c-19b5-461a-b579-ad6b0b80bd83\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:03:29.866915 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.866745    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p6js\" (UniqueName: \"kubernetes.io/projected/9812ee9c-19b5-461a-b579-ad6b0b80bd83-kube-api-access-9p6js\") pod \"payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4\" (UID: \"9812ee9c-19b5-461a-b579-ad6b0b80bd83\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:03:29.967875 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.967827    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9812ee9c-19b5-461a-b579-ad6b0b80bd83-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4\" (UID: \"9812ee9c-19b5-461a-b579-ad6b0b80bd83\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:03:29.968043 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.967889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e1be602-2676-490c-95f1-9111b7ba99ed-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n\" (UID: \"0e1be602-2676-490c-95f1-9111b7ba99ed\") " pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:03:29.968043 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.967929    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p6js\" (UniqueName: \"kubernetes.io/projected/9812ee9c-19b5-461a-b579-ad6b0b80bd83-kube-api-access-9p6js\") pod \"payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4\" (UID: \"9812ee9c-19b5-461a-b579-ad6b0b80bd83\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:03:29.968043 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.967966    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhqd6\" (UniqueName: \"kubernetes.io/projected/0e1be602-2676-490c-95f1-9111b7ba99ed-kube-api-access-fhqd6\") pod \"payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n\" (UID: \"0e1be602-2676-490c-95f1-9111b7ba99ed\") " pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:03:29.968639 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.968622    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9812ee9c-19b5-461a-b579-ad6b0b80bd83-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4\" (UID: \"9812ee9c-19b5-461a-b579-ad6b0b80bd83\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:03:29.985404 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:29.985335    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p6js\" (UniqueName: \"kubernetes.io/projected/9812ee9c-19b5-461a-b579-ad6b0b80bd83-kube-api-access-9p6js\") pod \"payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4\" (UID: \"9812ee9c-19b5-461a-b579-ad6b0b80bd83\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:03:30.035640 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.035548    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:03:30.061007 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.060745    2582 generic.go:356] "Generic (PLEG): container finished" podID="42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" containerID="4c8c6cc6fdcf245bed453c1833d5a655cf1c54509013ab4629cc8605b9fcc590" exitCode=0
Sep 11 18:03:30.061264 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.061225    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw" event={"ID":"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b","Type":"ContainerDied","Data":"4c8c6cc6fdcf245bed453c1833d5a655cf1c54509013ab4629cc8605b9fcc590"}
Sep 11 18:03:30.065233 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.065207    2582 generic.go:356] "Generic (PLEG): container finished" podID="78f62f94-9e42-4bca-bd2c-898cd8d1c807" containerID="0f7bff50d2ed0356d83571c0ef05aef25b3af2845cc9c97e387a82a6f2b52731" exitCode=0
Sep 11 18:03:30.065471 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.065450    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm" event={"ID":"78f62f94-9e42-4bca-bd2c-898cd8d1c807","Type":"ContainerDied","Data":"0f7bff50d2ed0356d83571c0ef05aef25b3af2845cc9c97e387a82a6f2b52731"}
Sep 11 18:03:30.068364 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.068333    2582 generic.go:356] "Generic (PLEG): container finished" podID="e144d1b2-48c2-4d3a-97da-99d4c9ee411b" containerID="a3dbfa233da1627d2977d70525b9e3f45d279cb6f8a62bc89585dec677436f1a" exitCode=0
Sep 11 18:03:30.068505 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.068441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8" event={"ID":"e144d1b2-48c2-4d3a-97da-99d4c9ee411b","Type":"ContainerDied","Data":"a3dbfa233da1627d2977d70525b9e3f45d279cb6f8a62bc89585dec677436f1a"}
Sep 11 18:03:30.069473 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.069448    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhqd6\" (UniqueName: \"kubernetes.io/projected/0e1be602-2676-490c-95f1-9111b7ba99ed-kube-api-access-fhqd6\") pod \"payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n\" (UID: \"0e1be602-2676-490c-95f1-9111b7ba99ed\") " pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:03:30.071141 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.070867    2582 generic.go:356] "Generic (PLEG): container finished" podID="4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" containerID="7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4" exitCode=0
Sep 11 18:03:30.071141 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.070913    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk" event={"ID":"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770","Type":"ContainerDied","Data":"7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4"}
Sep 11 18:03:30.071141 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.070941    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk" event={"ID":"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770","Type":"ContainerDied","Data":"7a1984a86259c49df9a0c254b88525b3176fb13ca5265e491f78312af936b613"}
Sep 11 18:03:30.071141 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.070951    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e1be602-2676-490c-95f1-9111b7ba99ed-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n\" (UID: \"0e1be602-2676-490c-95f1-9111b7ba99ed\") " pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:03:30.071141 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.070962    2582 scope.go:117] "RemoveContainer" containerID="7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4"
Sep 11 18:03:30.071141 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.071009    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"
Sep 11 18:03:30.071772 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.069842    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e1be602-2676-490c-95f1-9111b7ba99ed-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n\" (UID: \"0e1be602-2676-490c-95f1-9111b7ba99ed\") " pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:03:30.081654 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.081613    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhqd6\" (UniqueName: \"kubernetes.io/projected/0e1be602-2676-490c-95f1-9111b7ba99ed-kube-api-access-fhqd6\") pod \"payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n\" (UID: \"0e1be602-2676-490c-95f1-9111b7ba99ed\") " pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:03:30.084239 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.084025    2582 scope.go:117] "RemoveContainer" containerID="7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4"
Sep 11 18:03:30.084889 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:03:30.084823    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4\": container with ID starting with 7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4 not found: ID does not exist" containerID="7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4"
Sep 11 18:03:30.084889 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.084863    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4"} err="failed to get container status \"7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4\": rpc error: code = NotFound desc = could not find container \"7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4\": container with ID starting with 7eaa8627697fb7e05f59c4dfce1e63852f92a5d73e5040e950a2bd2a008d53c4 not found: ID does not exist"
Sep 11 18:03:30.095546 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.095526    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 18:03:30.100585 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.100565    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 18:03:30.105240 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.105223    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 18:03:30.173235 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173207    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lf7ms\" (UniqueName: \"kubernetes.io/projected/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-kube-api-access-lf7ms\") pod \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") "
Sep 11 18:03:30.173401 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173286    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2txwx\" (UniqueName: \"kubernetes.io/projected/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-kube-api-access-2txwx\") pod \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") "
Sep 11 18:03:30.173464 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173423    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tmp-dir\") pod \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") "
Sep 11 18:03:30.173753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173462    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-dshm\") pod \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") "
Sep 11 18:03:30.173753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173494    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-home\") pod \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") "
Sep 11 18:03:30.173753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173534    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-model-cache\") pod \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") "
Sep 11 18:03:30.173753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173556    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-home\") pod \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") "
Sep 11 18:03:30.173753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173623    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tls-certs\") pod \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") "
Sep 11 18:03:30.173753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173648    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tmp-dir\") pod \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") "
Sep 11 18:03:30.173753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173687    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tls-certs\") pod \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\" (UID: \"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b\") "
Sep 11 18:03:30.173753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173718    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-model-cache\") pod \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") "
Sep 11 18:03:30.173753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173752    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-dshm\") pod \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\" (UID: \"4ca43de3-32ec-4ad1-b0ba-4e0b695f3770\") "
Sep 11 18:03:30.174257 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.173837    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" (UID: "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.174257 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.174102    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-home" (OuterVolumeSpecName: "home") pod "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" (UID: "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.174462 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.174438    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tmp-dir\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.174537 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.174466    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-home\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.175024 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.174791    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-home" (OuterVolumeSpecName: "home") pod "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" (UID: "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.175024 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.174843    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-model-cache" (OuterVolumeSpecName: "model-cache") pod "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" (UID: "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.175024 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.174990    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-model-cache" (OuterVolumeSpecName: "model-cache") pod "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" (UID: "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.175638 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.175614    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" (UID: "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.176469 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.176441    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-kube-api-access-2txwx" (OuterVolumeSpecName: "kube-api-access-2txwx") pod "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" (UID: "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b"). InnerVolumeSpecName "kube-api-access-2txwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:03:30.176571 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.176478    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-kube-api-access-lf7ms" (OuterVolumeSpecName: "kube-api-access-lf7ms") pod "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" (UID: "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770"). InnerVolumeSpecName "kube-api-access-lf7ms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:03:30.177011 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.176984    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" (UID: "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:03:30.177831 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.177795    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" (UID: "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:03:30.177921 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.177886    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-dshm" (OuterVolumeSpecName: "dshm") pod "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" (UID: "4ca43de3-32ec-4ad1-b0ba-4e0b695f3770"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.177921 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.177880    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-dshm" (OuterVolumeSpecName: "dshm") pod "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" (UID: "42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.223274 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.223238    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:03:30.229411 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.229389    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.274955    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tls-certs\") pod \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275003    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tmp-dir\") pod \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275054    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-model-cache\") pod \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275119    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-dshm\") pod \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275141    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-home\") pod \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275189    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tmp-dir\") pod \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275230    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tm6p\" (UniqueName: \"kubernetes.io/projected/78f62f94-9e42-4bca-bd2c-898cd8d1c807-kube-api-access-2tm6p\") pod \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275263    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tls-certs\") pod \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275318    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq6gd\" (UniqueName: \"kubernetes.io/projected/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-kube-api-access-nq6gd\") pod \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275348    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-home\") pod \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275371    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-dshm\") pod \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\" (UID: \"e144d1b2-48c2-4d3a-97da-99d4c9ee411b\") "
Sep 11 18:03:30.275504 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275414    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-model-cache\") pod \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\" (UID: \"78f62f94-9e42-4bca-bd2c-898cd8d1c807\") "
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275765    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf7ms\" (UniqueName: \"kubernetes.io/projected/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-kube-api-access-lf7ms\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275784    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2txwx\" (UniqueName: \"kubernetes.io/projected/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-kube-api-access-2txwx\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275798    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-dshm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275811    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-home\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275823    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-model-cache\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275835    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tls-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275846    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-tmp-dir\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275856    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b-tls-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275867    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-model-cache\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.275880    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770-dshm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.276243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.276147    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-model-cache" (OuterVolumeSpecName: "model-cache") pod "78f62f94-9e42-4bca-bd2c-898cd8d1c807" (UID: "78f62f94-9e42-4bca-bd2c-898cd8d1c807"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.276885 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.276311    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "78f62f94-9e42-4bca-bd2c-898cd8d1c807" (UID: "78f62f94-9e42-4bca-bd2c-898cd8d1c807"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.276885 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.276365    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-model-cache" (OuterVolumeSpecName: "model-cache") pod "e144d1b2-48c2-4d3a-97da-99d4c9ee411b" (UID: "e144d1b2-48c2-4d3a-97da-99d4c9ee411b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.277492 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.277061    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-home" (OuterVolumeSpecName: "home") pod "e144d1b2-48c2-4d3a-97da-99d4c9ee411b" (UID: "e144d1b2-48c2-4d3a-97da-99d4c9ee411b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.278263 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.277948    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e144d1b2-48c2-4d3a-97da-99d4c9ee411b" (UID: "e144d1b2-48c2-4d3a-97da-99d4c9ee411b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.278263 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.278128    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-home" (OuterVolumeSpecName: "home") pod "78f62f94-9e42-4bca-bd2c-898cd8d1c807" (UID: "78f62f94-9e42-4bca-bd2c-898cd8d1c807"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.280348 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.279829    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78f62f94-9e42-4bca-bd2c-898cd8d1c807-kube-api-access-2tm6p" (OuterVolumeSpecName: "kube-api-access-2tm6p") pod "78f62f94-9e42-4bca-bd2c-898cd8d1c807" (UID: "78f62f94-9e42-4bca-bd2c-898cd8d1c807"). InnerVolumeSpecName "kube-api-access-2tm6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:03:30.280817 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.280471    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e144d1b2-48c2-4d3a-97da-99d4c9ee411b" (UID: "e144d1b2-48c2-4d3a-97da-99d4c9ee411b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:03:30.280817 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.280599    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-kube-api-access-nq6gd" (OuterVolumeSpecName: "kube-api-access-nq6gd") pod "e144d1b2-48c2-4d3a-97da-99d4c9ee411b" (UID: "e144d1b2-48c2-4d3a-97da-99d4c9ee411b"). InnerVolumeSpecName "kube-api-access-nq6gd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:03:30.282555 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.281303    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-dshm" (OuterVolumeSpecName: "dshm") pod "78f62f94-9e42-4bca-bd2c-898cd8d1c807" (UID: "78f62f94-9e42-4bca-bd2c-898cd8d1c807"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.285918 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.285872    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "78f62f94-9e42-4bca-bd2c-898cd8d1c807" (UID: "78f62f94-9e42-4bca-bd2c-898cd8d1c807"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:03:30.286480 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.286442    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-dshm" (OuterVolumeSpecName: "dshm") pod "e144d1b2-48c2-4d3a-97da-99d4c9ee411b" (UID: "e144d1b2-48c2-4d3a-97da-99d4c9ee411b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.376784 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376742    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tm6p\" (UniqueName: \"kubernetes.io/projected/78f62f94-9e42-4bca-bd2c-898cd8d1c807-kube-api-access-2tm6p\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.376784 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376774    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tls-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.376784 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376785    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq6gd\" (UniqueName: \"kubernetes.io/projected/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-kube-api-access-nq6gd\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.376784 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376793    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-home\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.377186 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376804    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-dshm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.377186 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376811    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-model-cache\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.377186 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376820    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tls-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.377186 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376827    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-tmp-dir\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.377186 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376834    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-model-cache\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.377186 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376841    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-dshm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.377186 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376847    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e144d1b2-48c2-4d3a-97da-99d4c9ee411b-home\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.377186 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.376854    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f62f94-9e42-4bca-bd2c-898cd8d1c807-tmp-dir\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.394262 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.394231    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"]
Sep 11 18:03:30.396239 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.396219    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/embedding-w0-286c27-kserve-5fc655b746-zrbkk"]
Sep 11 18:03:30.497814 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.497725    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"]
Sep 11 18:03:30.500781 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:03:30.500753    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9812ee9c_19b5_461a_b579_ad6b0b80bd83.slice/crio-aa5946625671cc9257913392434626adf5d48b7416214fee0c86e294c5de8bec WatchSource:0}: Error finding container aa5946625671cc9257913392434626adf5d48b7416214fee0c86e294c5de8bec: Status 404 returned error can't find the container with id aa5946625671cc9257913392434626adf5d48b7416214fee0c86e294c5de8bec
Sep 11 18:03:30.509937 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.509913    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"]
Sep 11 18:03:30.514813 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:03:30.514789    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1be602_2676_490c_95f1_9111b7ba99ed.slice/crio-a778443f8a9d43252c5e2ba076e99d4707be2d70cf6cf14b4364ce14b3f81f69 WatchSource:0}: Error finding container a778443f8a9d43252c5e2ba076e99d4707be2d70cf6cf14b4364ce14b3f81f69: Status 404 returned error can't find the container with id a778443f8a9d43252c5e2ba076e99d4707be2d70cf6cf14b4364ce14b3f81f69
Sep 11 18:03:30.636939 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:30.636905    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" path="/var/lib/kubelet/pods/4ca43de3-32ec-4ad1-b0ba-4e0b695f3770/volumes"
Sep 11 18:03:31.078716 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.078609    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"
Sep 11 18:03:31.078716 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.078639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw" event={"ID":"42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b","Type":"ContainerDied","Data":"d1c0f9216f214ee3386eba8608a10a0b202fdf0a5d55890e098c51f897eff19b"}
Sep 11 18:03:31.078716 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.078701    2582 scope.go:117] "RemoveContainer" containerID="4c8c6cc6fdcf245bed453c1833d5a655cf1c54509013ab4629cc8605b9fcc590"
Sep 11 18:03:31.080500 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.080471    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n" event={"ID":"0e1be602-2676-490c-95f1-9111b7ba99ed","Type":"ContainerStarted","Data":"fd6b16170ec0bedb862292000945409c76188eb6c46dde5eab56d6d5b5f657f7"}
Sep 11 18:03:31.080637 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.080538    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n" event={"ID":"0e1be602-2676-490c-95f1-9111b7ba99ed","Type":"ContainerStarted","Data":"a778443f8a9d43252c5e2ba076e99d4707be2d70cf6cf14b4364ce14b3f81f69"}
Sep 11 18:03:31.080637 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.080601    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:03:31.082565 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.082549    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"
Sep 11 18:03:31.082686 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.082583    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm" event={"ID":"78f62f94-9e42-4bca-bd2c-898cd8d1c807","Type":"ContainerDied","Data":"1d2b9e5ef7c5353ee9bb509869df28dfd56b62d38b6169bd8e913d7c7b5b683c"}
Sep 11 18:03:31.084033 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.084013    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"
Sep 11 18:03:31.084178 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.084052    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8" event={"ID":"e144d1b2-48c2-4d3a-97da-99d4c9ee411b","Type":"ContainerDied","Data":"018ef90ca3a8aa5ce0d4e98ea3c88a4e80da295f4902b869d9c6e3062a8ecdc7"}
Sep 11 18:03:31.085918 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.085894    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4" event={"ID":"9812ee9c-19b5-461a-b579-ad6b0b80bd83","Type":"ContainerStarted","Data":"4e44e8332f144fb3b0fae3819da143222b9e89d6b75409076928598f8f7f87de"}
Sep 11 18:03:31.086028 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.085938    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4" event={"ID":"9812ee9c-19b5-461a-b579-ad6b0b80bd83","Type":"ContainerStarted","Data":"aa5946625671cc9257913392434626adf5d48b7416214fee0c86e294c5de8bec"}
Sep 11 18:03:31.086028 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.085995    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:03:31.089514 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.089497    2582 scope.go:117] "RemoveContainer" containerID="0f7bff50d2ed0356d83571c0ef05aef25b3af2845cc9c97e387a82a6f2b52731"
Sep 11 18:03:31.097588 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.097572    2582 scope.go:117] "RemoveContainer" containerID="a3dbfa233da1627d2977d70525b9e3f45d279cb6f8a62bc89585dec677436f1a"
Sep 11 18:03:31.111553 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.111504    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n" podStartSLOduration=2.111490095 podStartE2EDuration="2.111490095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:03:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:03:31.110356054 +0000 UTC m=+1478.998534400" watchObservedRunningTime="2026-09-11 18:03:31.111490095 +0000 UTC m=+1478.999668472"
Sep 11 18:03:31.126016 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.125988    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"]
Sep 11 18:03:31.129004 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.128983    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/premium-w0-286c27-kserve-7f57df4c58-d9fpw"]
Sep 11 18:03:31.144732 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.144707    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"]
Sep 11 18:03:31.149459 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.149437    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/simulator-w0-286c27-kserve-6bc9f584dc-zz5zm"]
Sep 11 18:03:31.165513 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.165471    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4" podStartSLOduration=2.165457214 podStartE2EDuration="2.165457214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:03:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:03:31.164039929 +0000 UTC m=+1479.052218275" watchObservedRunningTime="2026-09-11 18:03:31.165457214 +0000 UTC m=+1479.053635565"
Sep 11 18:03:31.180766 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.180739    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"]
Sep 11 18:03:31.185733 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:31.185710    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-w0-286c27-kserve-785c8844f7-m86z8"]
Sep 11 18:03:32.636247 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:32.636216    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" path="/var/lib/kubelet/pods/42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b/volumes"
Sep 11 18:03:32.636612 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:32.636578    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78f62f94-9e42-4bca-bd2c-898cd8d1c807" path="/var/lib/kubelet/pods/78f62f94-9e42-4bca-bd2c-898cd8d1c807/volumes"
Sep 11 18:03:32.636916 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:32.636902    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e144d1b2-48c2-4d3a-97da-99d4c9ee411b" path="/var/lib/kubelet/pods/e144d1b2-48c2-4d3a-97da-99d4c9ee411b/volumes"
Sep 11 18:03:42.096782 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:42.096748    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:03:42.097223 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:03:42.097136    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:04:36.277058 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.277017    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"]
Sep 11 18:04:36.277671 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.277319    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n" podUID="0e1be602-2676-490c-95f1-9111b7ba99ed" containerName="payload-processing" containerID="cri-o://fd6b16170ec0bedb862292000945409c76188eb6c46dde5eab56d6d5b5f657f7" gracePeriod=30
Sep 11 18:04:36.283948 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.283684    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"]
Sep 11 18:04:36.283948 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.283891    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4" podUID="9812ee9c-19b5-461a-b579-ad6b0b80bd83" containerName="payload-pre-processing" containerID="cri-o://4e44e8332f144fb3b0fae3819da143222b9e89d6b75409076928598f8f7f87de" gracePeriod=30
Sep 11 18:04:36.426944 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.426902    2582 generic.go:356] "Generic (PLEG): container finished" podID="9812ee9c-19b5-461a-b579-ad6b0b80bd83" containerID="4e44e8332f144fb3b0fae3819da143222b9e89d6b75409076928598f8f7f87de" exitCode=0
Sep 11 18:04:36.427173 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.426969    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4" event={"ID":"9812ee9c-19b5-461a-b579-ad6b0b80bd83","Type":"ContainerDied","Data":"4e44e8332f144fb3b0fae3819da143222b9e89d6b75409076928598f8f7f87de"}
Sep 11 18:04:36.428930 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.428898    2582 generic.go:356] "Generic (PLEG): container finished" podID="0e1be602-2676-490c-95f1-9111b7ba99ed" containerID="fd6b16170ec0bedb862292000945409c76188eb6c46dde5eab56d6d5b5f657f7" exitCode=0
Sep 11 18:04:36.429068 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.428970    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n" event={"ID":"0e1be602-2676-490c-95f1-9111b7ba99ed","Type":"ContainerDied","Data":"fd6b16170ec0bedb862292000945409c76188eb6c46dde5eab56d6d5b5f657f7"}
Sep 11 18:04:36.543108 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.543018    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:04:36.569894 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.569868    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:04:36.605612 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.605572    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e1be602-2676-490c-95f1-9111b7ba99ed-plugins-config-volume\") pod \"0e1be602-2676-490c-95f1-9111b7ba99ed\" (UID: \"0e1be602-2676-490c-95f1-9111b7ba99ed\") "
Sep 11 18:04:36.605805 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.605639    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9p6js\" (UniqueName: \"kubernetes.io/projected/9812ee9c-19b5-461a-b579-ad6b0b80bd83-kube-api-access-9p6js\") pod \"9812ee9c-19b5-461a-b579-ad6b0b80bd83\" (UID: \"9812ee9c-19b5-461a-b579-ad6b0b80bd83\") "
Sep 11 18:04:36.605805 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.605677    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhqd6\" (UniqueName: \"kubernetes.io/projected/0e1be602-2676-490c-95f1-9111b7ba99ed-kube-api-access-fhqd6\") pod \"0e1be602-2676-490c-95f1-9111b7ba99ed\" (UID: \"0e1be602-2676-490c-95f1-9111b7ba99ed\") "
Sep 11 18:04:36.605805 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.605735    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9812ee9c-19b5-461a-b579-ad6b0b80bd83-plugins-config-volume\") pod \"9812ee9c-19b5-461a-b579-ad6b0b80bd83\" (UID: \"9812ee9c-19b5-461a-b579-ad6b0b80bd83\") "
Sep 11 18:04:36.606182 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.606149    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1be602-2676-490c-95f1-9111b7ba99ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0e1be602-2676-490c-95f1-9111b7ba99ed" (UID: "0e1be602-2676-490c-95f1-9111b7ba99ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:36.606299 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.606224    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9812ee9c-19b5-461a-b579-ad6b0b80bd83-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9812ee9c-19b5-461a-b579-ad6b0b80bd83" (UID: "9812ee9c-19b5-461a-b579-ad6b0b80bd83"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:36.608320 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.608284    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1be602-2676-490c-95f1-9111b7ba99ed-kube-api-access-fhqd6" (OuterVolumeSpecName: "kube-api-access-fhqd6") pod "0e1be602-2676-490c-95f1-9111b7ba99ed" (UID: "0e1be602-2676-490c-95f1-9111b7ba99ed"). InnerVolumeSpecName "kube-api-access-fhqd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:36.608438 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.608347    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9812ee9c-19b5-461a-b579-ad6b0b80bd83-kube-api-access-9p6js" (OuterVolumeSpecName: "kube-api-access-9p6js") pod "9812ee9c-19b5-461a-b579-ad6b0b80bd83" (UID: "9812ee9c-19b5-461a-b579-ad6b0b80bd83"). InnerVolumeSpecName "kube-api-access-9p6js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:36.707581 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.707466    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e1be602-2676-490c-95f1-9111b7ba99ed-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:36.707581 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.707502    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p6js\" (UniqueName: \"kubernetes.io/projected/9812ee9c-19b5-461a-b579-ad6b0b80bd83-kube-api-access-9p6js\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:36.707581 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.707516    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhqd6\" (UniqueName: \"kubernetes.io/projected/0e1be602-2676-490c-95f1-9111b7ba99ed-kube-api-access-fhqd6\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:36.707581 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:36.707530    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9812ee9c-19b5-461a-b579-ad6b0b80bd83-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:37.436564 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:37.436532    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"
Sep 11 18:04:37.437040 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:37.436564    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4" event={"ID":"9812ee9c-19b5-461a-b579-ad6b0b80bd83","Type":"ContainerDied","Data":"aa5946625671cc9257913392434626adf5d48b7416214fee0c86e294c5de8bec"}
Sep 11 18:04:37.437040 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:37.436634    2582 scope.go:117] "RemoveContainer" containerID="4e44e8332f144fb3b0fae3819da143222b9e89d6b75409076928598f8f7f87de"
Sep 11 18:04:37.437960 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:37.437935    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n" event={"ID":"0e1be602-2676-490c-95f1-9111b7ba99ed","Type":"ContainerDied","Data":"a778443f8a9d43252c5e2ba076e99d4707be2d70cf6cf14b4364ce14b3f81f69"}
Sep 11 18:04:37.437960 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:37.437955    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"
Sep 11 18:04:37.447821 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:37.447634    2582 scope.go:117] "RemoveContainer" containerID="fd6b16170ec0bedb862292000945409c76188eb6c46dde5eab56d6d5b5f657f7"
Sep 11 18:04:37.454412 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:37.454380    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"]
Sep 11 18:04:37.460031 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:37.460001    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a93550-87dc4b787-qdp6n"]
Sep 11 18:04:37.470187 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:37.470154    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"]
Sep 11 18:04:37.478943 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:37.478912    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a93550-5ffddd6d54-dkcz4"]
Sep 11 18:04:38.594040 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:38.593994    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"]
Sep 11 18:04:38.635240 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:38.635204    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e1be602-2676-490c-95f1-9111b7ba99ed" path="/var/lib/kubelet/pods/0e1be602-2676-490c-95f1-9111b7ba99ed/volumes"
Sep 11 18:04:38.635635 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:38.635620    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9812ee9c-19b5-461a-b579-ad6b0b80bd83" path="/var/lib/kubelet/pods/9812ee9c-19b5-461a-b579-ad6b0b80bd83/volumes"
Sep 11 18:04:50.181751 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.181719    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k"]
Sep 11 18:04:50.186150 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.186118    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-4ab253-29819160-64m2k"]
Sep 11 18:04:50.226676 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.226632    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"]
Sep 11 18:04:50.226888 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.226859    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l" podUID="ab22c37f-c2ac-4247-bef9-48a84d2f02ca" containerName="payload-pre-processing" containerID="cri-o://f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d" gracePeriod=30
Sep 11 18:04:50.492585 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.492557    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 18:04:50.509130 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.509074    2582 generic.go:356] "Generic (PLEG): container finished" podID="ab22c37f-c2ac-4247-bef9-48a84d2f02ca" containerID="f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d" exitCode=0
Sep 11 18:04:50.509130 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.509123    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l" event={"ID":"ab22c37f-c2ac-4247-bef9-48a84d2f02ca","Type":"ContainerDied","Data":"f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d"}
Sep 11 18:04:50.509413 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.509154    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"
Sep 11 18:04:50.509413 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.509190    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l" event={"ID":"ab22c37f-c2ac-4247-bef9-48a84d2f02ca","Type":"ContainerDied","Data":"b48c7c52f4a73dfb5115070d61d609f72e374fb1bd074251eb27e5f52c2d081d"}
Sep 11 18:04:50.509413 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.509229    2582 scope.go:117] "RemoveContainer" containerID="f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d"
Sep 11 18:04:50.522237 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.522216    2582 scope.go:117] "RemoveContainer" containerID="f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d"
Sep 11 18:04:50.522565 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:04:50.522542    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d\": container with ID starting with f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d not found: ID does not exist" containerID="f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d"
Sep 11 18:04:50.522674 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.522574    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d"} err="failed to get container status \"f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d\": rpc error: code = NotFound desc = could not find container \"f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d\": container with ID starting with f8a2dcb8280a054804e99c7b67f630e56b5ba66fc63f8aad9e73c126fca4d05d not found: ID does not exist"
Sep 11 18:04:50.634945 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.634913    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d934f1c-22d2-4575-9541-9d6d85965d24" path="/var/lib/kubelet/pods/6d934f1c-22d2-4575-9541-9d6d85965d24/volumes"
Sep 11 18:04:50.635165 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.635145    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-plugins-config-volume\") pod \"ab22c37f-c2ac-4247-bef9-48a84d2f02ca\" (UID: \"ab22c37f-c2ac-4247-bef9-48a84d2f02ca\") "
Sep 11 18:04:50.635249 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.635185    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p986b\" (UniqueName: \"kubernetes.io/projected/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-kube-api-access-p986b\") pod \"ab22c37f-c2ac-4247-bef9-48a84d2f02ca\" (UID: \"ab22c37f-c2ac-4247-bef9-48a84d2f02ca\") "
Sep 11 18:04:50.635559 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.635539    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab22c37f-c2ac-4247-bef9-48a84d2f02ca" (UID: "ab22c37f-c2ac-4247-bef9-48a84d2f02ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:50.637339 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.637317    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-kube-api-access-p986b" (OuterVolumeSpecName: "kube-api-access-p986b") pod "ab22c37f-c2ac-4247-bef9-48a84d2f02ca" (UID: "ab22c37f-c2ac-4247-bef9-48a84d2f02ca"). InnerVolumeSpecName "kube-api-access-p986b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:50.736147 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.736038    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:50.736147 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.736070    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p986b\" (UniqueName: \"kubernetes.io/projected/ab22c37f-c2ac-4247-bef9-48a84d2f02ca-kube-api-access-p986b\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:50.833203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.833165    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"]
Sep 11 18:04:50.838411 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:50.838382    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-4ab253-54b95c96d6-2l98l"]
Sep 11 18:04:52.634720 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:52.634689    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab22c37f-c2ac-4247-bef9-48a84d2f02ca" path="/var/lib/kubelet/pods/ab22c37f-c2ac-4247-bef9-48a84d2f02ca/volumes"
Sep 11 18:04:53.614632 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.614590    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b" podUID="cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" containerName="main" containerID="cri-o://17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf" gracePeriod=45
Sep 11 18:04:53.873547 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.873487    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:04:53.967440 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.967401    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6p2wk\" (UniqueName: \"kubernetes.io/projected/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-kube-api-access-6p2wk\") pod \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") "
Sep 11 18:04:53.967645 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.967471    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tmp-dir\") pod \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") "
Sep 11 18:04:53.967645 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.967537    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-home\") pod \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") "
Sep 11 18:04:53.967645 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.967594    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-dshm\") pod \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") "
Sep 11 18:04:53.967645 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.967624    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-model-cache\") pod \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") "
Sep 11 18:04:53.967845 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.967678    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tls-certs\") pod \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\" (UID: \"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57\") "
Sep 11 18:04:53.967845 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.967724    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" (UID: "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:53.967934 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.967872    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-home" (OuterVolumeSpecName: "home") pod "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" (UID: "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:53.968293 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.968055    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-model-cache" (OuterVolumeSpecName: "model-cache") pod "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" (UID: "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:53.968293 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.968059    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tmp-dir\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:53.968293 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.968128    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-home\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:53.969956 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.969930    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-dshm" (OuterVolumeSpecName: "dshm") pod "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" (UID: "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:53.970472 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.970440    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" (UID: "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:04:53.970472 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:53.970453    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-kube-api-access-6p2wk" (OuterVolumeSpecName: "kube-api-access-6p2wk") pod "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" (UID: "cf9c353c-735a-4f8e-aea4-6dd3ebd66b57"). InnerVolumeSpecName "kube-api-access-6p2wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:54.069621 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.069581    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-dshm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.069621 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.069612    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-model-cache\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.069621 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.069621    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-tls-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.069621 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.069631    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p2wk\" (UniqueName: \"kubernetes.io/projected/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57-kube-api-access-6p2wk\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.373874 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.373831    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"]
Sep 11 18:04:54.374112 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.374068    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj" podUID="2fcb9b7c-2081-4dad-8dbc-0bec01466729" containerName="maas-api" containerID="cri-o://5ff505fc804df7e664cae54254631bd03d6b2744175234cb06c3189b133fa6b0" gracePeriod=30
Sep 11 18:04:54.436244 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.436207    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"]
Sep 11 18:04:54.436494 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.436464    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz" podUID="3809c931-66aa-4e3a-80b2-7a25334317db" containerName="payload-processing" containerID="cri-o://401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa" gracePeriod=30
Sep 11 18:04:54.534422 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.534385    2582 generic.go:356] "Generic (PLEG): container finished" podID="2fcb9b7c-2081-4dad-8dbc-0bec01466729" containerID="5ff505fc804df7e664cae54254631bd03d6b2744175234cb06c3189b133fa6b0" exitCode=0
Sep 11 18:04:54.534575 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.534455    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj" event={"ID":"2fcb9b7c-2081-4dad-8dbc-0bec01466729","Type":"ContainerDied","Data":"5ff505fc804df7e664cae54254631bd03d6b2744175234cb06c3189b133fa6b0"}
Sep 11 18:04:54.535778 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.535745    2582 generic.go:356] "Generic (PLEG): container finished" podID="cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" containerID="17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf" exitCode=0
Sep 11 18:04:54.535778 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.535779    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b" event={"ID":"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57","Type":"ContainerDied","Data":"17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf"}
Sep 11 18:04:54.535953 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.535791    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"
Sep 11 18:04:54.535953 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.535801    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b" event={"ID":"cf9c353c-735a-4f8e-aea4-6dd3ebd66b57","Type":"ContainerDied","Data":"0c66977cb92080cf3eceee763a9b3d222bef92c8ff92f794a82e9eda5fafc109"}
Sep 11 18:04:54.535953 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.535816    2582 scope.go:117] "RemoveContainer" containerID="17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf"
Sep 11 18:04:54.549237 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.549210    2582 scope.go:117] "RemoveContainer" containerID="17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf"
Sep 11 18:04:54.549610 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:04:54.549582    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf\": container with ID starting with 17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf not found: ID does not exist" containerID="17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf"
Sep 11 18:04:54.549703 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.549622    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf"} err="failed to get container status \"17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf\": rpc error: code = NotFound desc = could not find container \"17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf\": container with ID starting with 17e5c17ccd86afdbe455307c6f0b6d4512d2cdd37e3305b747ae068e6d1edddf not found: ID does not exist"
Sep 11 18:04:54.562827 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.562800    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"]
Sep 11 18:04:54.566653 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.566615    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/distinct-w4-4ab253-kserve-54dbc46cc6-bv85b"]
Sep 11 18:04:54.635974 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.635934    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" path="/var/lib/kubelet/pods/cf9c353c-735a-4f8e-aea4-6dd3ebd66b57/volumes"
Sep 11 18:04:54.691688 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.691657    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 18:04:54.722978 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.722948    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 18:04:54.776537 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.776510    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-maas-api-tls\") pod \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") "
Sep 11 18:04:54.776719 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.776576    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcb9b7c-2081-4dad-8dbc-0bec01466729-openshift-service-ca\") pod \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") "
Sep 11 18:04:54.776719 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.776639    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5bw6\" (UniqueName: \"kubernetes.io/projected/2fcb9b7c-2081-4dad-8dbc-0bec01466729-kube-api-access-k5bw6\") pod \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") "
Sep 11 18:04:54.776719 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.776680    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-metrics-certs\") pod \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\" (UID: \"2fcb9b7c-2081-4dad-8dbc-0bec01466729\") "
Sep 11 18:04:54.776989 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.776956    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fcb9b7c-2081-4dad-8dbc-0bec01466729-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2fcb9b7c-2081-4dad-8dbc-0bec01466729" (UID: "2fcb9b7c-2081-4dad-8dbc-0bec01466729"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:54.778791 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.778767    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2fcb9b7c-2081-4dad-8dbc-0bec01466729" (UID: "2fcb9b7c-2081-4dad-8dbc-0bec01466729"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:04:54.779333 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.779312    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2fcb9b7c-2081-4dad-8dbc-0bec01466729" (UID: "2fcb9b7c-2081-4dad-8dbc-0bec01466729"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:04:54.779407 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.779361    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fcb9b7c-2081-4dad-8dbc-0bec01466729-kube-api-access-k5bw6" (OuterVolumeSpecName: "kube-api-access-k5bw6") pod "2fcb9b7c-2081-4dad-8dbc-0bec01466729" (UID: "2fcb9b7c-2081-4dad-8dbc-0bec01466729"). InnerVolumeSpecName "kube-api-access-k5bw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:54.878191 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.878154    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3809c931-66aa-4e3a-80b2-7a25334317db-plugins-config-volume\") pod \"3809c931-66aa-4e3a-80b2-7a25334317db\" (UID: \"3809c931-66aa-4e3a-80b2-7a25334317db\") "
Sep 11 18:04:54.878616 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.878279    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47rvp\" (UniqueName: \"kubernetes.io/projected/3809c931-66aa-4e3a-80b2-7a25334317db-kube-api-access-47rvp\") pod \"3809c931-66aa-4e3a-80b2-7a25334317db\" (UID: \"3809c931-66aa-4e3a-80b2-7a25334317db\") "
Sep 11 18:04:54.878616 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.878559    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-maas-api-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.878616 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.878579    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcb9b7c-2081-4dad-8dbc-0bec01466729-openshift-service-ca\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.878616 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.878595    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5bw6\" (UniqueName: \"kubernetes.io/projected/2fcb9b7c-2081-4dad-8dbc-0bec01466729-kube-api-access-k5bw6\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.878616 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.878608    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fcb9b7c-2081-4dad-8dbc-0bec01466729-metrics-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.878616 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.878606    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3809c931-66aa-4e3a-80b2-7a25334317db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3809c931-66aa-4e3a-80b2-7a25334317db" (UID: "3809c931-66aa-4e3a-80b2-7a25334317db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:54.880445 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.880392    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3809c931-66aa-4e3a-80b2-7a25334317db-kube-api-access-47rvp" (OuterVolumeSpecName: "kube-api-access-47rvp") pod "3809c931-66aa-4e3a-80b2-7a25334317db" (UID: "3809c931-66aa-4e3a-80b2-7a25334317db"). InnerVolumeSpecName "kube-api-access-47rvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:54.979785 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.979744    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47rvp\" (UniqueName: \"kubernetes.io/projected/3809c931-66aa-4e3a-80b2-7a25334317db-kube-api-access-47rvp\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.979785 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:54.979780    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3809c931-66aa-4e3a-80b2-7a25334317db-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:04:55.547190 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.547149    2582 generic.go:356] "Generic (PLEG): container finished" podID="3809c931-66aa-4e3a-80b2-7a25334317db" containerID="401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa" exitCode=0
Sep 11 18:04:55.547410 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.547216    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"
Sep 11 18:04:55.547410 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.547236    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz" event={"ID":"3809c931-66aa-4e3a-80b2-7a25334317db","Type":"ContainerDied","Data":"401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa"}
Sep 11 18:04:55.547410 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.547318    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz" event={"ID":"3809c931-66aa-4e3a-80b2-7a25334317db","Type":"ContainerDied","Data":"1ef54f12855fe02fcb9bed2e842652ad1dd6419d495716e10fa7456f73455412"}
Sep 11 18:04:55.547410 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.547350    2582 scope.go:117] "RemoveContainer" containerID="401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa"
Sep 11 18:04:55.548862 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.548822    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"
Sep 11 18:04:55.548862 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.548826    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj" event={"ID":"2fcb9b7c-2081-4dad-8dbc-0bec01466729","Type":"ContainerDied","Data":"7cf711a484cbc5d2632bda6196735e7522168cc8d97e692a1f273baf01038638"}
Sep 11 18:04:55.558454 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.557405    2582 scope.go:117] "RemoveContainer" containerID="401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa"
Sep 11 18:04:55.558454 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:04:55.557787    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa\": container with ID starting with 401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa not found: ID does not exist" containerID="401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa"
Sep 11 18:04:55.558454 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.557820    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa"} err="failed to get container status \"401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa\": rpc error: code = NotFound desc = could not find container \"401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa\": container with ID starting with 401bf6045fc2b890de397ab9cb896553a7e750ee3c9cada2e1ce27d2abf3aafa not found: ID does not exist"
Sep 11 18:04:55.558454 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.557844    2582 scope.go:117] "RemoveContainer" containerID="5ff505fc804df7e664cae54254631bd03d6b2744175234cb06c3189b133fa6b0"
Sep 11 18:04:55.576312 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.576283    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"]
Sep 11 18:04:55.580652 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.580624    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-21e472-79f78c5467-pgdlz"]
Sep 11 18:04:55.595518 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.595489    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"]
Sep 11 18:04:55.599172 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:55.599147    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-21e472-7cc594bb76-p5pjj"]
Sep 11 18:04:56.634190 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:56.634158    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fcb9b7c-2081-4dad-8dbc-0bec01466729" path="/var/lib/kubelet/pods/2fcb9b7c-2081-4dad-8dbc-0bec01466729/volumes"
Sep 11 18:04:56.634588 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:04:56.634522    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3809c931-66aa-4e3a-80b2-7a25334317db" path="/var/lib/kubelet/pods/3809c931-66aa-4e3a-80b2-7a25334317db/volumes"
Sep 11 18:05:12.712923 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:12.712888    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th"]
Sep 11 18:05:12.716992 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:12.716965    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-7f85c3-29819160-cg6th"]
Sep 11 18:05:12.736310 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:12.736274    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"]
Sep 11 18:05:12.736536 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:12.736508    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd" podUID="dc0563ad-e78f-46a6-aded-fb4de8f60e3a" containerName="payload-processing" containerID="cri-o://3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a" gracePeriod=30
Sep 11 18:05:12.748734 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:12.748704    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"]
Sep 11 18:05:12.748995 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:12.748965    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759" podUID="3dc40db8-4669-4479-a9ec-c5865f42d394" containerName="payload-pre-processing" containerID="cri-o://6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50" gracePeriod=30
Sep 11 18:05:12.996162 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:12.996138    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 18:05:13.029830 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.029806    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 18:05:13.045815 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.045784    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdtsg\" (UniqueName: \"kubernetes.io/projected/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-kube-api-access-pdtsg\") pod \"dc0563ad-e78f-46a6-aded-fb4de8f60e3a\" (UID: \"dc0563ad-e78f-46a6-aded-fb4de8f60e3a\") "
Sep 11 18:05:13.046003 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.045831    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-plugins-config-volume\") pod \"dc0563ad-e78f-46a6-aded-fb4de8f60e3a\" (UID: \"dc0563ad-e78f-46a6-aded-fb4de8f60e3a\") "
Sep 11 18:05:13.046003 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.045920    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dc40db8-4669-4479-a9ec-c5865f42d394-plugins-config-volume\") pod \"3dc40db8-4669-4479-a9ec-c5865f42d394\" (UID: \"3dc40db8-4669-4479-a9ec-c5865f42d394\") "
Sep 11 18:05:13.046003 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.045964    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c62v4\" (UniqueName: \"kubernetes.io/projected/3dc40db8-4669-4479-a9ec-c5865f42d394-kube-api-access-c62v4\") pod \"3dc40db8-4669-4479-a9ec-c5865f42d394\" (UID: \"3dc40db8-4669-4479-a9ec-c5865f42d394\") "
Sep 11 18:05:13.046343 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.046310    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc0563ad-e78f-46a6-aded-fb4de8f60e3a" (UID: "dc0563ad-e78f-46a6-aded-fb4de8f60e3a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:05:13.046479 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.046336    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dc40db8-4669-4479-a9ec-c5865f42d394-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3dc40db8-4669-4479-a9ec-c5865f42d394" (UID: "3dc40db8-4669-4479-a9ec-c5865f42d394"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:05:13.048125 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.048102    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-kube-api-access-pdtsg" (OuterVolumeSpecName: "kube-api-access-pdtsg") pod "dc0563ad-e78f-46a6-aded-fb4de8f60e3a" (UID: "dc0563ad-e78f-46a6-aded-fb4de8f60e3a"). InnerVolumeSpecName "kube-api-access-pdtsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:05:13.048275 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.048250    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dc40db8-4669-4479-a9ec-c5865f42d394-kube-api-access-c62v4" (OuterVolumeSpecName: "kube-api-access-c62v4") pod "3dc40db8-4669-4479-a9ec-c5865f42d394" (UID: "3dc40db8-4669-4479-a9ec-c5865f42d394"). InnerVolumeSpecName "kube-api-access-c62v4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:05:13.147421 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.147375    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdtsg\" (UniqueName: \"kubernetes.io/projected/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-kube-api-access-pdtsg\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:05:13.147421 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.147411    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc0563ad-e78f-46a6-aded-fb4de8f60e3a-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:05:13.147421 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.147421    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dc40db8-4669-4479-a9ec-c5865f42d394-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:05:13.147421 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.147429    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c62v4\" (UniqueName: \"kubernetes.io/projected/3dc40db8-4669-4479-a9ec-c5865f42d394-kube-api-access-c62v4\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:05:13.645484 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.645446    2582 generic.go:356] "Generic (PLEG): container finished" podID="dc0563ad-e78f-46a6-aded-fb4de8f60e3a" containerID="3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a" exitCode=0
Sep 11 18:05:13.645679 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.645490    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"
Sep 11 18:05:13.645679 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.645535    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd" event={"ID":"dc0563ad-e78f-46a6-aded-fb4de8f60e3a","Type":"ContainerDied","Data":"3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a"}
Sep 11 18:05:13.645679 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.645583    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd" event={"ID":"dc0563ad-e78f-46a6-aded-fb4de8f60e3a","Type":"ContainerDied","Data":"50c2d3d0e73933661c4ff766275486579d3e32d7ab8f4b9d3a2dff77d35591b0"}
Sep 11 18:05:13.645679 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.645608    2582 scope.go:117] "RemoveContainer" containerID="3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a"
Sep 11 18:05:13.646981 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.646958    2582 generic.go:356] "Generic (PLEG): container finished" podID="3dc40db8-4669-4479-a9ec-c5865f42d394" containerID="6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50" exitCode=0
Sep 11 18:05:13.647118 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.647009    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"
Sep 11 18:05:13.647118 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.647045    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759" event={"ID":"3dc40db8-4669-4479-a9ec-c5865f42d394","Type":"ContainerDied","Data":"6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50"}
Sep 11 18:05:13.647118 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.647116    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759" event={"ID":"3dc40db8-4669-4479-a9ec-c5865f42d394","Type":"ContainerDied","Data":"5d702dc0e558edf2faad66c6a2b997cb0193644eae457d10c9659f90696cf84f"}
Sep 11 18:05:13.654749 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.654730    2582 scope.go:117] "RemoveContainer" containerID="3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a"
Sep 11 18:05:13.655166 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:05:13.655053    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a\": container with ID starting with 3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a not found: ID does not exist" containerID="3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a"
Sep 11 18:05:13.655166 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.655095    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a"} err="failed to get container status \"3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a\": rpc error: code = NotFound desc = could not find container \"3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a\": container with ID starting with 3774e87d6b43f4b984edb2dffed5d8be529fd88f2be5f9039c55fdbff755889a not found: ID does not exist"
Sep 11 18:05:13.655166 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.655118    2582 scope.go:117] "RemoveContainer" containerID="6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50"
Sep 11 18:05:13.667070 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.667045    2582 scope.go:117] "RemoveContainer" containerID="6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50"
Sep 11 18:05:13.667439 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:05:13.667421    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50\": container with ID starting with 6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50 not found: ID does not exist" containerID="6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50"
Sep 11 18:05:13.667501 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.667446    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50"} err="failed to get container status \"6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50\": rpc error: code = NotFound desc = could not find container \"6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50\": container with ID starting with 6079bee2a16e1aa894e16fc9d722698ee2cf27d7216ed5bf6ec45cd1542d2e50 not found: ID does not exist"
Sep 11 18:05:13.677144 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.677112    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"]
Sep 11 18:05:13.681460 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.681435    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7f85c3-5c48ff96c-2hwxd"]
Sep 11 18:05:13.693526 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.693495    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"]
Sep 11 18:05:13.697319 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:13.697287    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7f85c3-7f98f84bc4-t2759"]
Sep 11 18:05:14.634163 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:14.634128    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dc40db8-4669-4479-a9ec-c5865f42d394" path="/var/lib/kubelet/pods/3dc40db8-4669-4479-a9ec-c5865f42d394/volumes"
Sep 11 18:05:14.634527 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:14.634450    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50dc8939-7afd-4a93-b984-304954611f62" path="/var/lib/kubelet/pods/50dc8939-7afd-4a93-b984-304954611f62/volumes"
Sep 11 18:05:14.634776 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:14.634761    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc0563ad-e78f-46a6-aded-fb4de8f60e3a" path="/var/lib/kubelet/pods/dc0563ad-e78f-46a6-aded-fb4de8f60e3a/volumes"
Sep 11 18:05:19.468782 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.468743    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8096-63f961"]
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469113    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" containerName="main"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469128    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" containerName="main"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469138    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1be602-2676-490c-95f1-9111b7ba99ed" containerName="payload-processing"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469144    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1be602-2676-490c-95f1-9111b7ba99ed" containerName="payload-processing"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469153    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab22c37f-c2ac-4247-bef9-48a84d2f02ca" containerName="payload-pre-processing"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469159    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab22c37f-c2ac-4247-bef9-48a84d2f02ca" containerName="payload-pre-processing"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469170    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78f62f94-9e42-4bca-bd2c-898cd8d1c807" containerName="main"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469175    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="78f62f94-9e42-4bca-bd2c-898cd8d1c807" containerName="main"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469180    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" containerName="main"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469184    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" containerName="main"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469191    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3809c931-66aa-4e3a-80b2-7a25334317db" containerName="payload-processing"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469197    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3809c931-66aa-4e3a-80b2-7a25334317db" containerName="payload-processing"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469204    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc0563ad-e78f-46a6-aded-fb4de8f60e3a" containerName="payload-processing"
Sep 11 18:05:19.469203 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469209    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc0563ad-e78f-46a6-aded-fb4de8f60e3a" containerName="payload-processing"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469217    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" containerName="main"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469222    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" containerName="main"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469230    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9812ee9c-19b5-461a-b579-ad6b0b80bd83" containerName="payload-pre-processing"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469236    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9812ee9c-19b5-461a-b579-ad6b0b80bd83" containerName="payload-pre-processing"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469242    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dc40db8-4669-4479-a9ec-c5865f42d394" containerName="payload-pre-processing"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469247    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dc40db8-4669-4479-a9ec-c5865f42d394" containerName="payload-pre-processing"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469255    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e144d1b2-48c2-4d3a-97da-99d4c9ee411b" containerName="main"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469260    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e144d1b2-48c2-4d3a-97da-99d4c9ee411b" containerName="main"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469267    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fcb9b7c-2081-4dad-8dbc-0bec01466729" containerName="maas-api"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469271    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fcb9b7c-2081-4dad-8dbc-0bec01466729" containerName="maas-api"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469322    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3809c931-66aa-4e3a-80b2-7a25334317db" containerName="payload-processing"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469331    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab22c37f-c2ac-4247-bef9-48a84d2f02ca" containerName="payload-pre-processing"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469337    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1be602-2676-490c-95f1-9111b7ba99ed" containerName="payload-processing"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469342    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="78f62f94-9e42-4bca-bd2c-898cd8d1c807" containerName="main"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469347    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9812ee9c-19b5-461a-b579-ad6b0b80bd83" containerName="payload-pre-processing"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469351    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="42de3dc5-f6d7-48df-9fd1-9d8d6cdbc83b" containerName="main"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469356    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e144d1b2-48c2-4d3a-97da-99d4c9ee411b" containerName="main"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469360    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fcb9b7c-2081-4dad-8dbc-0bec01466729" containerName="maas-api"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469367    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dc40db8-4669-4479-a9ec-c5865f42d394" containerName="payload-pre-processing"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469372    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf9c353c-735a-4f8e-aea4-6dd3ebd66b57" containerName="main"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469378    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca43de3-32ec-4ad1-b0ba-4e0b695f3770" containerName="main"
Sep 11 18:05:19.469629 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.469383    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc0563ad-e78f-46a6-aded-fb4de8f60e3a" containerName="payload-processing"
Sep 11 18:05:19.473675 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.473657    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-63f961"
Sep 11 18:05:19.477125 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.477074    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zsx9b\""
Sep 11 18:05:19.485109 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.485054    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8096-63f961"]
Sep 11 18:05:19.505557 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.505526    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr6rb\" (UniqueName: \"kubernetes.io/projected/34f8952e-964b-4278-ab7c-a9c29af1d6de-kube-api-access-hr6rb\") pod \"test-curl-8096-63f961\" (UID: \"34f8952e-964b-4278-ab7c-a9c29af1d6de\") " pod="opendatahub/test-curl-8096-63f961"
Sep 11 18:05:19.606409 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.606365    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr6rb\" (UniqueName: \"kubernetes.io/projected/34f8952e-964b-4278-ab7c-a9c29af1d6de-kube-api-access-hr6rb\") pod \"test-curl-8096-63f961\" (UID: \"34f8952e-964b-4278-ab7c-a9c29af1d6de\") " pod="opendatahub/test-curl-8096-63f961"
Sep 11 18:05:19.622964 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.622929    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr6rb\" (UniqueName: \"kubernetes.io/projected/34f8952e-964b-4278-ab7c-a9c29af1d6de-kube-api-access-hr6rb\") pod \"test-curl-8096-63f961\" (UID: \"34f8952e-964b-4278-ab7c-a9c29af1d6de\") " pod="opendatahub/test-curl-8096-63f961"
Sep 11 18:05:19.783098 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:19.783007    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-63f961"
Sep 11 18:05:20.066701 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:20.066677    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8096-63f961"]
Sep 11 18:05:20.069235 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:05:20.069204    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34f8952e_964b_4278_ab7c_a9c29af1d6de.slice/crio-3cdc0fec432866132531b2b1531644671b8141b9462b4550c27489fe2131bec9 WatchSource:0}: Error finding container 3cdc0fec432866132531b2b1531644671b8141b9462b4550c27489fe2131bec9: Status 404 returned error can't find the container with id 3cdc0fec432866132531b2b1531644671b8141b9462b4550c27489fe2131bec9
Sep 11 18:05:20.685753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:20.685703    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8096-63f961" event={"ID":"34f8952e-964b-4278-ab7c-a9c29af1d6de","Type":"ContainerStarted","Data":"973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976"}
Sep 11 18:05:20.685753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:20.685756    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8096-63f961" event={"ID":"34f8952e-964b-4278-ab7c-a9c29af1d6de","Type":"ContainerStarted","Data":"3cdc0fec432866132531b2b1531644671b8141b9462b4550c27489fe2131bec9"}
Sep 11 18:05:20.708665 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:20.708609    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8096-63f961" podStartSLOduration=1.7085896969999999 podStartE2EDuration="1.708589697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:05:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:05:20.704627923 +0000 UTC m=+1588.592806269" watchObservedRunningTime="2026-09-11 18:05:20.708589697 +0000 UTC m=+1588.596768083"
Sep 11 18:05:21.419575 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:21.419539    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8096-63f961"]
Sep 11 18:05:21.426505 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:21.426470    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8096-63f961"]
Sep 11 18:05:21.691298 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:05:21.691266    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8096-63f961" container="test-curl-8096-63f961"
Sep 11 18:05:22.695422 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:22.695380    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8096-63f961" podUID="34f8952e-964b-4278-ab7c-a9c29af1d6de" containerName="test-curl-8096-63f961" containerID="cri-o://973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976" gracePeriod=2
Sep 11 18:05:24.884177 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:24.884152    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-63f961"
Sep 11 18:05:24.960740 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:24.960646    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr6rb\" (UniqueName: \"kubernetes.io/projected/34f8952e-964b-4278-ab7c-a9c29af1d6de-kube-api-access-hr6rb\") pod \"34f8952e-964b-4278-ab7c-a9c29af1d6de\" (UID: \"34f8952e-964b-4278-ab7c-a9c29af1d6de\") "
Sep 11 18:05:24.962857 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:24.962832    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34f8952e-964b-4278-ab7c-a9c29af1d6de-kube-api-access-hr6rb" (OuterVolumeSpecName: "kube-api-access-hr6rb") pod "34f8952e-964b-4278-ab7c-a9c29af1d6de" (UID: "34f8952e-964b-4278-ab7c-a9c29af1d6de"). InnerVolumeSpecName "kube-api-access-hr6rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:05:25.061757 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:25.061711    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr6rb\" (UniqueName: \"kubernetes.io/projected/34f8952e-964b-4278-ab7c-a9c29af1d6de-kube-api-access-hr6rb\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:05:25.712466 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:25.712425    2582 generic.go:356] "Generic (PLEG): container finished" podID="34f8952e-964b-4278-ab7c-a9c29af1d6de" containerID="973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976" exitCode=137
Sep 11 18:05:25.712625 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:25.712473    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-63f961"
Sep 11 18:05:25.712625 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:25.712508    2582 scope.go:117] "RemoveContainer" containerID="973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976"
Sep 11 18:05:25.723597 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:25.723572    2582 scope.go:117] "RemoveContainer" containerID="973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976"
Sep 11 18:05:25.723731 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:05:25.723642    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8096-63f961\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="34f8952e-964b-4278-ab7c-a9c29af1d6de" pod="opendatahub/test-curl-8096-63f961"
Sep 11 18:05:25.723974 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:05:25.723937    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976\": container with ID starting with 973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976 not found: ID does not exist" containerID="973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976"
Sep 11 18:05:25.724041 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:25.723992    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976"} err="failed to get container status \"973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976\": rpc error: code = NotFound desc = could not find container \"973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976\": container with ID starting with 973e448b33b38327cd70d49808d4e851f83097583b813ffe913df26a5ab66976 not found: ID does not exist"
Sep 11 18:05:26.633825 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:05:26.633788    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34f8952e-964b-4278-ab7c-a9c29af1d6de" path="/var/lib/kubelet/pods/34f8952e-964b-4278-ab7c-a9c29af1d6de/volumes"
Sep 11 18:06:49.322124 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.322072    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8096-37d2cd"]
Sep 11 18:06:49.322539 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.322469    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34f8952e-964b-4278-ab7c-a9c29af1d6de" containerName="test-curl-8096-63f961"
Sep 11 18:06:49.322539 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.322481    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="34f8952e-964b-4278-ab7c-a9c29af1d6de" containerName="test-curl-8096-63f961"
Sep 11 18:06:49.322622 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.322547    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="34f8952e-964b-4278-ab7c-a9c29af1d6de" containerName="test-curl-8096-63f961"
Sep 11 18:06:49.325437 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.325422    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-37d2cd"
Sep 11 18:06:49.328692 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.328673    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zsx9b\""
Sep 11 18:06:49.347174 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.347144    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8096-37d2cd"]
Sep 11 18:06:49.435159 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.435124    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkrp9\" (UniqueName: \"kubernetes.io/projected/47ef50d5-7f10-45d7-b62e-9c92b32d8649-kube-api-access-bkrp9\") pod \"test-curl-8096-37d2cd\" (UID: \"47ef50d5-7f10-45d7-b62e-9c92b32d8649\") " pod="opendatahub/test-curl-8096-37d2cd"
Sep 11 18:06:49.536284 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.536233    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkrp9\" (UniqueName: \"kubernetes.io/projected/47ef50d5-7f10-45d7-b62e-9c92b32d8649-kube-api-access-bkrp9\") pod \"test-curl-8096-37d2cd\" (UID: \"47ef50d5-7f10-45d7-b62e-9c92b32d8649\") " pod="opendatahub/test-curl-8096-37d2cd"
Sep 11 18:06:49.548190 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.548163    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkrp9\" (UniqueName: \"kubernetes.io/projected/47ef50d5-7f10-45d7-b62e-9c92b32d8649-kube-api-access-bkrp9\") pod \"test-curl-8096-37d2cd\" (UID: \"47ef50d5-7f10-45d7-b62e-9c92b32d8649\") " pod="opendatahub/test-curl-8096-37d2cd"
Sep 11 18:06:49.634670 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.634579    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-37d2cd"
Sep 11 18:06:49.912586 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:49.912562    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8096-37d2cd"]
Sep 11 18:06:49.914144 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:06:49.914116    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47ef50d5_7f10_45d7_b62e_9c92b32d8649.slice/crio-5913f586ec4db686f45cff475b376de0b23f523786175571cab65c7a4e783a50 WatchSource:0}: Error finding container 5913f586ec4db686f45cff475b376de0b23f523786175571cab65c7a4e783a50: Status 404 returned error can't find the container with id 5913f586ec4db686f45cff475b376de0b23f523786175571cab65c7a4e783a50
Sep 11 18:06:50.149689 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:50.149656    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8096-37d2cd" event={"ID":"47ef50d5-7f10-45d7-b62e-9c92b32d8649","Type":"ContainerStarted","Data":"a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff"}
Sep 11 18:06:50.149689 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:50.149694    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8096-37d2cd" event={"ID":"47ef50d5-7f10-45d7-b62e-9c92b32d8649","Type":"ContainerStarted","Data":"5913f586ec4db686f45cff475b376de0b23f523786175571cab65c7a4e783a50"}
Sep 11 18:06:50.186904 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:50.186790    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8096-37d2cd" podStartSLOduration=1.18676998 podStartE2EDuration="1.18676998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:06:50.184626145 +0000 UTC m=+1678.072804491" watchObservedRunningTime="2026-09-11 18:06:50.18676998 +0000 UTC m=+1678.074948394"
Sep 11 18:06:50.931066 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:50.930986    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8096-37d2cd"]
Sep 11 18:06:50.949803 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:50.949765    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8096-37d2cd"]
Sep 11 18:06:51.083798 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.083760    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8096-38cc54"]
Sep 11 18:06:51.084163 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.084150    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47ef50d5-7f10-45d7-b62e-9c92b32d8649" containerName="test-curl-8096-37d2cd"
Sep 11 18:06:51.084214 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.084165    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="47ef50d5-7f10-45d7-b62e-9c92b32d8649" containerName="test-curl-8096-37d2cd"
Sep 11 18:06:51.084250 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.084226    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="47ef50d5-7f10-45d7-b62e-9c92b32d8649" containerName="test-curl-8096-37d2cd"
Sep 11 18:06:51.087266 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.087247    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-38cc54"
Sep 11 18:06:51.096836 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.096811    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8096-38cc54"]
Sep 11 18:06:51.148583 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.148539    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tktmb\" (UniqueName: \"kubernetes.io/projected/058eb157-076b-4b1f-b302-d5d09b8ae775-kube-api-access-tktmb\") pod \"test-curl-8096-38cc54\" (UID: \"058eb157-076b-4b1f-b302-d5d09b8ae775\") " pod="opendatahub/test-curl-8096-38cc54"
Sep 11 18:06:51.154616 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:06:51.154581    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8096-37d2cd" container="test-curl-8096-37d2cd"
Sep 11 18:06:51.249734 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.249631    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tktmb\" (UniqueName: \"kubernetes.io/projected/058eb157-076b-4b1f-b302-d5d09b8ae775-kube-api-access-tktmb\") pod \"test-curl-8096-38cc54\" (UID: \"058eb157-076b-4b1f-b302-d5d09b8ae775\") " pod="opendatahub/test-curl-8096-38cc54"
Sep 11 18:06:51.260908 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.260882    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tktmb\" (UniqueName: \"kubernetes.io/projected/058eb157-076b-4b1f-b302-d5d09b8ae775-kube-api-access-tktmb\") pod \"test-curl-8096-38cc54\" (UID: \"058eb157-076b-4b1f-b302-d5d09b8ae775\") " pod="opendatahub/test-curl-8096-38cc54"
Sep 11 18:06:51.397326 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.397294    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-38cc54"
Sep 11 18:06:51.670381 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:51.670349    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8096-38cc54"]
Sep 11 18:06:51.671694 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:06:51.671667    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod058eb157_076b_4b1f_b302_d5d09b8ae775.slice/crio-f56aaf3f35ffc13278d0a954b5435053133e042b92f4f4d4c4e89a72d706e8a5 WatchSource:0}: Error finding container f56aaf3f35ffc13278d0a954b5435053133e042b92f4f4d4c4e89a72d706e8a5: Status 404 returned error can't find the container with id f56aaf3f35ffc13278d0a954b5435053133e042b92f4f4d4c4e89a72d706e8a5
Sep 11 18:06:52.161095 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:52.161055    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8096-38cc54" event={"ID":"058eb157-076b-4b1f-b302-d5d09b8ae775","Type":"ContainerStarted","Data":"0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07"}
Sep 11 18:06:52.161534 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:52.161128    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8096-38cc54" event={"ID":"058eb157-076b-4b1f-b302-d5d09b8ae775","Type":"ContainerStarted","Data":"f56aaf3f35ffc13278d0a954b5435053133e042b92f4f4d4c4e89a72d706e8a5"}
Sep 11 18:06:52.161534 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:52.161248    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8096-37d2cd" podUID="47ef50d5-7f10-45d7-b62e-9c92b32d8649" containerName="test-curl-8096-37d2cd" containerID="cri-o://a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff" gracePeriod=2
Sep 11 18:06:52.182803 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:52.182755    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8096-38cc54" podStartSLOduration=1.1827394390000001 podStartE2EDuration="1.182739439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:06:52.181494686 +0000 UTC m=+1680.069673031" watchObservedRunningTime="2026-09-11 18:06:52.182739439 +0000 UTC m=+1680.070917784"
Sep 11 18:06:52.184301 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:06:52.184274    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8096-37d2cd\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="47ef50d5-7f10-45d7-b62e-9c92b32d8649" pod="opendatahub/test-curl-8096-37d2cd"
Sep 11 18:06:52.633259 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:06:52.633226    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8096-37d2cd\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="47ef50d5-7f10-45d7-b62e-9c92b32d8649" pod="opendatahub/test-curl-8096-37d2cd"
Sep 11 18:06:52.844764 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:52.844731    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8096-38cc54"]
Sep 11 18:06:52.851746 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:52.851715    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8096-38cc54"]
Sep 11 18:06:52.864002 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:52.863961    2582 scope.go:117] "RemoveContainer" containerID="fa7951f2812d66f995d4d828dbfb55b00f253a54ee10e94a1612c996da90f3e4"
Sep 11 18:06:52.873001 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:52.872986    2582 scope.go:117] "RemoveContainer" containerID="9725acbd1c6e18230ca95e3c9b9f0fab18c1ea2ead9529dff174efa15f5a3856"
Sep 11 18:06:53.165609 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:06:53.165582    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8096-38cc54" container="test-curl-8096-38cc54"
Sep 11 18:06:54.170315 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:54.170279    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8096-38cc54" podUID="058eb157-076b-4b1f-b302-d5d09b8ae775" containerName="test-curl-8096-38cc54" containerID="cri-o://0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07" gracePeriod=2
Sep 11 18:06:54.175031 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:06:54.175000    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8096-38cc54\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="058eb157-076b-4b1f-b302-d5d09b8ae775" pod="opendatahub/test-curl-8096-38cc54"
Sep 11 18:06:54.359386 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:54.359360    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-37d2cd"
Sep 11 18:06:54.362209 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:06:54.362185    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8096-38cc54\" is forbidden: User \"system:node:ip-10-0-140-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-215.ec2.internal' and this object" podUID="058eb157-076b-4b1f-b302-d5d09b8ae775" pod="opendatahub/test-curl-8096-38cc54"
Sep 11 18:06:54.477727 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:54.477638    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkrp9\" (UniqueName: \"kubernetes.io/projected/47ef50d5-7f10-45d7-b62e-9c92b32d8649-kube-api-access-bkrp9\") pod \"47ef50d5-7f10-45d7-b62e-9c92b32d8649\" (UID: \"47ef50d5-7f10-45d7-b62e-9c92b32d8649\") "
Sep 11 18:06:54.479891 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:54.479865    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47ef50d5-7f10-45d7-b62e-9c92b32d8649-kube-api-access-bkrp9" (OuterVolumeSpecName: "kube-api-access-bkrp9") pod "47ef50d5-7f10-45d7-b62e-9c92b32d8649" (UID: "47ef50d5-7f10-45d7-b62e-9c92b32d8649"). InnerVolumeSpecName "kube-api-access-bkrp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:06:54.578468 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:54.578434    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkrp9\" (UniqueName: \"kubernetes.io/projected/47ef50d5-7f10-45d7-b62e-9c92b32d8649-kube-api-access-bkrp9\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:06:54.634138 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:54.634103    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47ef50d5-7f10-45d7-b62e-9c92b32d8649" path="/var/lib/kubelet/pods/47ef50d5-7f10-45d7-b62e-9c92b32d8649/volumes"
Sep 11 18:06:55.175613 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:55.175575    2582 generic.go:356] "Generic (PLEG): container finished" podID="47ef50d5-7f10-45d7-b62e-9c92b32d8649" containerID="a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff" exitCode=137
Sep 11 18:06:55.175613 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:55.175616    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-37d2cd"
Sep 11 18:06:55.176148 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:55.175663    2582 scope.go:117] "RemoveContainer" containerID="a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff"
Sep 11 18:06:55.184001 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:55.183982    2582 scope.go:117] "RemoveContainer" containerID="a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff"
Sep 11 18:06:55.184315 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:06:55.184293    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff\": container with ID starting with a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff not found: ID does not exist" containerID="a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff"
Sep 11 18:06:55.184389 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:55.184325    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff"} err="failed to get container status \"a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff\": rpc error: code = NotFound desc = could not find container \"a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff\": container with ID starting with a35dceeb9d1cfa200a41d0ca05bd542b18bdd89ac37c0318b32563c21116fdff not found: ID does not exist"
Sep 11 18:06:56.348041 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:56.348013    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-38cc54"
Sep 11 18:06:56.395260 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:56.395217    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tktmb\" (UniqueName: \"kubernetes.io/projected/058eb157-076b-4b1f-b302-d5d09b8ae775-kube-api-access-tktmb\") pod \"058eb157-076b-4b1f-b302-d5d09b8ae775\" (UID: \"058eb157-076b-4b1f-b302-d5d09b8ae775\") "
Sep 11 18:06:56.397434 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:56.397409    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/058eb157-076b-4b1f-b302-d5d09b8ae775-kube-api-access-tktmb" (OuterVolumeSpecName: "kube-api-access-tktmb") pod "058eb157-076b-4b1f-b302-d5d09b8ae775" (UID: "058eb157-076b-4b1f-b302-d5d09b8ae775"). InnerVolumeSpecName "kube-api-access-tktmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:06:56.496053 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:56.495971    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tktmb\" (UniqueName: \"kubernetes.io/projected/058eb157-076b-4b1f-b302-d5d09b8ae775-kube-api-access-tktmb\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:06:56.633705 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:56.633668    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="058eb157-076b-4b1f-b302-d5d09b8ae775" path="/var/lib/kubelet/pods/058eb157-076b-4b1f-b302-d5d09b8ae775/volumes"
Sep 11 18:06:57.189370 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:57.189337    2582 generic.go:356] "Generic (PLEG): container finished" podID="058eb157-076b-4b1f-b302-d5d09b8ae775" containerID="0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07" exitCode=137
Sep 11 18:06:57.189573 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:57.189381    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8096-38cc54"
Sep 11 18:06:57.189573 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:57.189394    2582 scope.go:117] "RemoveContainer" containerID="0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07"
Sep 11 18:06:57.197611 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:57.197592    2582 scope.go:117] "RemoveContainer" containerID="0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07"
Sep 11 18:06:57.197904 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:06:57.197883    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07\": container with ID starting with 0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07 not found: ID does not exist" containerID="0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07"
Sep 11 18:06:57.197955 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:06:57.197916    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07"} err="failed to get container status \"0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07\": rpc error: code = NotFound desc = could not find container \"0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07\": container with ID starting with 0ff0b09ba574457167e8a3a8cf2b1e4b7c42bac7f32da7efd712bb5205aa5e07 not found: ID does not exist"
Sep 11 18:08:43.660559 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.660469    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"]
Sep 11 18:08:43.661028 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.660851    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="058eb157-076b-4b1f-b302-d5d09b8ae775" containerName="test-curl-8096-38cc54"
Sep 11 18:08:43.661028 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.660867    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="058eb157-076b-4b1f-b302-d5d09b8ae775" containerName="test-curl-8096-38cc54"
Sep 11 18:08:43.661028 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.660936    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="058eb157-076b-4b1f-b302-d5d09b8ae775" containerName="test-curl-8096-38cc54"
Sep 11 18:08:43.664234 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.664213    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.667360 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.667339    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-64600a-serving-cert\""
Sep 11 18:08:43.667464 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.667346    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:08:43.668416 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.668373    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 18:08:43.668503 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.668488    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5vrxn\""
Sep 11 18:08:43.668688 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.668675    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 18:08:43.676559 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.676535    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"]
Sep 11 18:08:43.794038 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.793997    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-metrics-certs\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.794274 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.794051    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/108dedb2-6c6e-4720-9be9-1ecda5457a65-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.794274 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.794202    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-maas-api-tls\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.794274 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.794237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6rrt\" (UniqueName: \"kubernetes.io/projected/108dedb2-6c6e-4720-9be9-1ecda5457a65-kube-api-access-b6rrt\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.895335 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.895300    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-metrics-certs\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.895511 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.895344    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/108dedb2-6c6e-4720-9be9-1ecda5457a65-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.895511 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.895401    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-maas-api-tls\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.895511 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.895423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6rrt\" (UniqueName: \"kubernetes.io/projected/108dedb2-6c6e-4720-9be9-1ecda5457a65-kube-api-access-b6rrt\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.896248 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.896158    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/108dedb2-6c6e-4720-9be9-1ecda5457a65-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.898064 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.898039    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-metrics-certs\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.898180 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.898128    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-maas-api-tls\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.907572 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.907550    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6rrt\" (UniqueName: \"kubernetes.io/projected/108dedb2-6c6e-4720-9be9-1ecda5457a65-kube-api-access-b6rrt\") pod \"maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:43.974767 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:43.974670    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:44.267333 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:44.267304    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"]
Sep 11 18:08:44.269964 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:08:44.269933    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108dedb2_6c6e_4720_9be9_1ecda5457a65.slice/crio-5cf29a3d6e9e6f3180f8d1547b735d14ecd988fd8d59dd7b3da8ba58131e5cd7 WatchSource:0}: Error finding container 5cf29a3d6e9e6f3180f8d1547b735d14ecd988fd8d59dd7b3da8ba58131e5cd7: Status 404 returned error can't find the container with id 5cf29a3d6e9e6f3180f8d1547b735d14ecd988fd8d59dd7b3da8ba58131e5cd7
Sep 11 18:08:44.271327 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:44.271311    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:08:44.711988 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:44.711946    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89" event={"ID":"108dedb2-6c6e-4720-9be9-1ecda5457a65","Type":"ContainerStarted","Data":"5cf29a3d6e9e6f3180f8d1547b735d14ecd988fd8d59dd7b3da8ba58131e5cd7"}
Sep 11 18:08:45.718269 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:45.718229    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89" event={"ID":"108dedb2-6c6e-4720-9be9-1ecda5457a65","Type":"ContainerStarted","Data":"1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3"}
Sep 11 18:08:45.718652 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:45.718307    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:08:45.745709 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:45.745661    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89" podStartSLOduration=2.28190021 podStartE2EDuration="2.745641775s" totalImagesPullingTime="463.741565ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:08:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:08:45.745274863 +0000 UTC m=+1793.633453210" watchObservedRunningTime="2026-09-11 18:08:45.745641775 +0000 UTC m=+1793.633820122"
Sep 11 18:08:51.728482 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:08:51.728452    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:09:15.087959 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.087925    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"]
Sep 11 18:09:15.091655 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.091637    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:09:15.097535 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.097507    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f8f6a0\""
Sep 11 18:09:15.100038 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.100012    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"]
Sep 11 18:09:15.103405 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.103385    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:09:15.106255 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.106228    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f8f6a0-dockercfg-795gs\""
Sep 11 18:09:15.119868 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.119819    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"]
Sep 11 18:09:15.165068 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.165033    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"]
Sep 11 18:09:15.173892 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.173858    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b68fb55-3665-47e4-a479-611f89437e11-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5\" (UID: \"9b68fb55-3665-47e4-a479-611f89437e11\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:09:15.174106 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.173929    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a21fe026-d4d4-4de8-b515-bb60bb173b3b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2\" (UID: \"a21fe026-d4d4-4de8-b515-bb60bb173b3b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:09:15.174106 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.173995    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4jxw\" (UniqueName: \"kubernetes.io/projected/9b68fb55-3665-47e4-a479-611f89437e11-kube-api-access-l4jxw\") pod \"payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5\" (UID: \"9b68fb55-3665-47e4-a479-611f89437e11\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:09:15.174106 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.174021    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngsxw\" (UniqueName: \"kubernetes.io/projected/a21fe026-d4d4-4de8-b515-bb60bb173b3b-kube-api-access-ngsxw\") pod \"payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2\" (UID: \"a21fe026-d4d4-4de8-b515-bb60bb173b3b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:09:15.274736 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.274700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jxw\" (UniqueName: \"kubernetes.io/projected/9b68fb55-3665-47e4-a479-611f89437e11-kube-api-access-l4jxw\") pod \"payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5\" (UID: \"9b68fb55-3665-47e4-a479-611f89437e11\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:09:15.274947 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.274752    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngsxw\" (UniqueName: \"kubernetes.io/projected/a21fe026-d4d4-4de8-b515-bb60bb173b3b-kube-api-access-ngsxw\") pod \"payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2\" (UID: \"a21fe026-d4d4-4de8-b515-bb60bb173b3b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:09:15.274947 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.274788    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b68fb55-3665-47e4-a479-611f89437e11-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5\" (UID: \"9b68fb55-3665-47e4-a479-611f89437e11\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:09:15.274947 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.274858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a21fe026-d4d4-4de8-b515-bb60bb173b3b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2\" (UID: \"a21fe026-d4d4-4de8-b515-bb60bb173b3b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:09:15.275554 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.275535    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a21fe026-d4d4-4de8-b515-bb60bb173b3b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2\" (UID: \"a21fe026-d4d4-4de8-b515-bb60bb173b3b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:09:15.275609 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.275551    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b68fb55-3665-47e4-a479-611f89437e11-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5\" (UID: \"9b68fb55-3665-47e4-a479-611f89437e11\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:09:15.286215 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.286185    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4jxw\" (UniqueName: \"kubernetes.io/projected/9b68fb55-3665-47e4-a479-611f89437e11-kube-api-access-l4jxw\") pod \"payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5\" (UID: \"9b68fb55-3665-47e4-a479-611f89437e11\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:09:15.286341 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.286317    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngsxw\" (UniqueName: \"kubernetes.io/projected/a21fe026-d4d4-4de8-b515-bb60bb173b3b-kube-api-access-ngsxw\") pod \"payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2\" (UID: \"a21fe026-d4d4-4de8-b515-bb60bb173b3b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:09:15.403423 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.403383    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:09:15.413672 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.413634    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:09:15.688017 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.687990    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"]
Sep 11 18:09:15.690158 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:09:15.690132    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda21fe026_d4d4_4de8_b515_bb60bb173b3b.slice/crio-ea974925eb7fde6331457e7f5c139db828dc8af9ecdcb17bbff03607a4399e1c WatchSource:0}: Error finding container ea974925eb7fde6331457e7f5c139db828dc8af9ecdcb17bbff03607a4399e1c: Status 404 returned error can't find the container with id ea974925eb7fde6331457e7f5c139db828dc8af9ecdcb17bbff03607a4399e1c
Sep 11 18:09:15.697397 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.697373    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"]
Sep 11 18:09:15.698809 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:09:15.698780    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b68fb55_3665_47e4_a479_611f89437e11.slice/crio-a50da8df8afe4c51f20d0338e618815b90a57d2c2af0d6172f5c9b2b52f76ea6 WatchSource:0}: Error finding container a50da8df8afe4c51f20d0338e618815b90a57d2c2af0d6172f5c9b2b52f76ea6: Status 404 returned error can't find the container with id a50da8df8afe4c51f20d0338e618815b90a57d2c2af0d6172f5c9b2b52f76ea6
Sep 11 18:09:15.868873 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.868830    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5" event={"ID":"9b68fb55-3665-47e4-a479-611f89437e11","Type":"ContainerStarted","Data":"b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b"}
Sep 11 18:09:15.869121 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.868986    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:09:15.869121 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.869027    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5" event={"ID":"9b68fb55-3665-47e4-a479-611f89437e11","Type":"ContainerStarted","Data":"a50da8df8afe4c51f20d0338e618815b90a57d2c2af0d6172f5c9b2b52f76ea6"}
Sep 11 18:09:15.870536 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.870509    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2" event={"ID":"a21fe026-d4d4-4de8-b515-bb60bb173b3b","Type":"ContainerStarted","Data":"faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095"}
Sep 11 18:09:15.870668 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.870557    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2" event={"ID":"a21fe026-d4d4-4de8-b515-bb60bb173b3b","Type":"ContainerStarted","Data":"ea974925eb7fde6331457e7f5c139db828dc8af9ecdcb17bbff03607a4399e1c"}
Sep 11 18:09:15.870668 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.870586    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:09:15.893974 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.893919    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5" podStartSLOduration=0.893900974 podStartE2EDuration="893.900974ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:09:15.892709275 +0000 UTC m=+1823.780887620" watchObservedRunningTime="2026-09-11 18:09:15.893900974 +0000 UTC m=+1823.782079320"
Sep 11 18:09:15.919476 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:15.919415    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2" podStartSLOduration=0.919392735 podStartE2EDuration="919.392735ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:09:15.918183479 +0000 UTC m=+1823.806361839" watchObservedRunningTime="2026-09-11 18:09:15.919392735 +0000 UTC m=+1823.807571083"
Sep 11 18:09:26.877794 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:26.877761    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:09:26.878332 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:26.877836    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:09:46.977777 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:09:46.977746    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5_9b68fb55-3665-47e4-a479-611f89437e11/payload-processing/0.log"
Sep 11 18:10:28.538942 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:28.538910    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5_9b68fb55-3665-47e4-a479-611f89437e11/payload-processing/0.log"
Sep 11 18:10:36.606899 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.606862    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"]
Sep 11 18:10:36.610805 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.610778    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.614387 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.614358    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-2db21a-serving-cert\""
Sep 11 18:10:36.636899 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.636867    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"]
Sep 11 18:10:36.740456 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.740416    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-482wm\" (UniqueName: \"kubernetes.io/projected/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-kube-api-access-482wm\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.740658 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.740468    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.740658 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.740632    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.740772 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.740668    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.841479 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.841441    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.841479 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.841481    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.841709 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.841513    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-482wm\" (UniqueName: \"kubernetes.io/projected/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-kube-api-access-482wm\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.841709 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.841539    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.841709 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:10:36.841645    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-serving-cert: secret "maas-api-e2e-ipp-cleanup-2db21a-serving-cert" not found
Sep 11 18:10:36.841866 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:10:36.841733    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-maas-api-tls podName:f1fa9d81-8da0-4d34-95f6-6d8db1e79178 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:37.341712671 +0000 UTC m=+1905.229890995 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8" (UID: "f1fa9d81-8da0-4d34-95f6-6d8db1e79178") : secret "maas-api-e2e-ipp-cleanup-2db21a-serving-cert" not found
Sep 11 18:10:36.842159 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.842140    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.844222 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.844197    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:36.852277 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:36.852254    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-482wm\" (UniqueName: \"kubernetes.io/projected/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-kube-api-access-482wm\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:37.346667 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:37.346630    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:37.349255 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:37.349232    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:37.521742 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:37.521693    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:37.799985 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:37.799952    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"]
Sep 11 18:10:37.802338 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:10:37.802307    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1fa9d81_8da0_4d34_95f6_6d8db1e79178.slice/crio-d6ab11e7d05d2c7fadb5f3154404b2103c9e99e2a438683976c36311894ae790 WatchSource:0}: Error finding container d6ab11e7d05d2c7fadb5f3154404b2103c9e99e2a438683976c36311894ae790: Status 404 returned error can't find the container with id d6ab11e7d05d2c7fadb5f3154404b2103c9e99e2a438683976c36311894ae790
Sep 11 18:10:38.280071 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:38.280033    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8" event={"ID":"f1fa9d81-8da0-4d34-95f6-6d8db1e79178","Type":"ContainerStarted","Data":"d6ab11e7d05d2c7fadb5f3154404b2103c9e99e2a438683976c36311894ae790"}
Sep 11 18:10:39.286121 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:39.286059    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8" event={"ID":"f1fa9d81-8da0-4d34-95f6-6d8db1e79178","Type":"ContainerStarted","Data":"f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326"}
Sep 11 18:10:39.286507 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:39.286153    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:10:39.320373 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:39.320313    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8" podStartSLOduration=2.839289806 podStartE2EDuration="3.320294893s" totalImagesPullingTime="481.005087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:10:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:39.319070142 +0000 UTC m=+1907.207248487" watchObservedRunningTime="2026-09-11 18:10:39.320294893 +0000 UTC m=+1907.208473261"
Sep 11 18:10:45.296476 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:10:45.296447    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:11:14.491236 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.491200    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"]
Sep 11 18:11:14.492051 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.491994    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8" podUID="f1fa9d81-8da0-4d34-95f6-6d8db1e79178" containerName="maas-api" containerID="cri-o://f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326" gracePeriod=30
Sep 11 18:11:14.746811 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.746750    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:11:14.900265 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.900236    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-openshift-service-ca\") pod \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") "
Sep 11 18:11:14.900462 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.900274    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-maas-api-tls\") pod \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") "
Sep 11 18:11:14.900462 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.900303    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-482wm\" (UniqueName: \"kubernetes.io/projected/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-kube-api-access-482wm\") pod \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") "
Sep 11 18:11:14.900462 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.900328    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-metrics-certs\") pod \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\" (UID: \"f1fa9d81-8da0-4d34-95f6-6d8db1e79178\") "
Sep 11 18:11:14.900690 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.900662    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1fa9d81-8da0-4d34-95f6-6d8db1e79178" (UID: "f1fa9d81-8da0-4d34-95f6-6d8db1e79178"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:11:14.902669 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.902642    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f1fa9d81-8da0-4d34-95f6-6d8db1e79178" (UID: "f1fa9d81-8da0-4d34-95f6-6d8db1e79178"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:11:14.902669 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.902661    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1fa9d81-8da0-4d34-95f6-6d8db1e79178" (UID: "f1fa9d81-8da0-4d34-95f6-6d8db1e79178"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:11:14.902849 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:14.902719    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-kube-api-access-482wm" (OuterVolumeSpecName: "kube-api-access-482wm") pod "f1fa9d81-8da0-4d34-95f6-6d8db1e79178" (UID: "f1fa9d81-8da0-4d34-95f6-6d8db1e79178"). InnerVolumeSpecName "kube-api-access-482wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:11:15.001765 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.001674    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-openshift-service-ca\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:11:15.001765 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.001706    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-maas-api-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:11:15.001765 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.001716    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-482wm\" (UniqueName: \"kubernetes.io/projected/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-kube-api-access-482wm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:11:15.001765 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.001724    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1fa9d81-8da0-4d34-95f6-6d8db1e79178-metrics-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:11:15.458795 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.458750    2582 generic.go:356] "Generic (PLEG): container finished" podID="f1fa9d81-8da0-4d34-95f6-6d8db1e79178" containerID="f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326" exitCode=0
Sep 11 18:11:15.458983 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.458797    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"
Sep 11 18:11:15.458983 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.458834    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8" event={"ID":"f1fa9d81-8da0-4d34-95f6-6d8db1e79178","Type":"ContainerDied","Data":"f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326"}
Sep 11 18:11:15.458983 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.458893    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8" event={"ID":"f1fa9d81-8da0-4d34-95f6-6d8db1e79178","Type":"ContainerDied","Data":"d6ab11e7d05d2c7fadb5f3154404b2103c9e99e2a438683976c36311894ae790"}
Sep 11 18:11:15.458983 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.458920    2582 scope.go:117] "RemoveContainer" containerID="f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326"
Sep 11 18:11:15.468975 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.468956    2582 scope.go:117] "RemoveContainer" containerID="f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326"
Sep 11 18:11:15.469356 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:11:15.469335    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326\": container with ID starting with f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326 not found: ID does not exist" containerID="f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326"
Sep 11 18:11:15.469417 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.469367    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326"} err="failed to get container status \"f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326\": rpc error: code = NotFound desc = could not find container \"f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326\": container with ID starting with f9b95e8090e6b2c38753950a8fa8561581c09b8f13a6837b288b1764e055d326 not found: ID does not exist"
Sep 11 18:11:15.505593 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.505557    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"]
Sep 11 18:11:15.511868 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:15.511835    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2db21a-66ddb65df8-42np8"]
Sep 11 18:11:16.633954 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:16.633921    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1fa9d81-8da0-4d34-95f6-6d8db1e79178" path="/var/lib/kubelet/pods/f1fa9d81-8da0-4d34-95f6-6d8db1e79178/volumes"
Sep 11 18:11:41.128129 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.128062    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"]
Sep 11 18:11:41.128524 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.128419    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89" podUID="108dedb2-6c6e-4720-9be9-1ecda5457a65" containerName="maas-api" containerID="cri-o://1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3" gracePeriod=30
Sep 11 18:11:41.393701 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.393674    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:11:41.441931 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.441889    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6rrt\" (UniqueName: \"kubernetes.io/projected/108dedb2-6c6e-4720-9be9-1ecda5457a65-kube-api-access-b6rrt\") pod \"108dedb2-6c6e-4720-9be9-1ecda5457a65\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") "
Sep 11 18:11:41.442136 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.441950    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-metrics-certs\") pod \"108dedb2-6c6e-4720-9be9-1ecda5457a65\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") "
Sep 11 18:11:41.442136 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.442037    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/108dedb2-6c6e-4720-9be9-1ecda5457a65-openshift-service-ca\") pod \"108dedb2-6c6e-4720-9be9-1ecda5457a65\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") "
Sep 11 18:11:41.442272 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.442163    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-maas-api-tls\") pod \"108dedb2-6c6e-4720-9be9-1ecda5457a65\" (UID: \"108dedb2-6c6e-4720-9be9-1ecda5457a65\") "
Sep 11 18:11:41.442440 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.442411    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/108dedb2-6c6e-4720-9be9-1ecda5457a65-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "108dedb2-6c6e-4720-9be9-1ecda5457a65" (UID: "108dedb2-6c6e-4720-9be9-1ecda5457a65"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:11:41.444441 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.444408    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "108dedb2-6c6e-4720-9be9-1ecda5457a65" (UID: "108dedb2-6c6e-4720-9be9-1ecda5457a65"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:11:41.444754 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.444727    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/108dedb2-6c6e-4720-9be9-1ecda5457a65-kube-api-access-b6rrt" (OuterVolumeSpecName: "kube-api-access-b6rrt") pod "108dedb2-6c6e-4720-9be9-1ecda5457a65" (UID: "108dedb2-6c6e-4720-9be9-1ecda5457a65"). InnerVolumeSpecName "kube-api-access-b6rrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:11:41.444754 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.444733    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "108dedb2-6c6e-4720-9be9-1ecda5457a65" (UID: "108dedb2-6c6e-4720-9be9-1ecda5457a65"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:11:41.543438 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.543403    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6rrt\" (UniqueName: \"kubernetes.io/projected/108dedb2-6c6e-4720-9be9-1ecda5457a65-kube-api-access-b6rrt\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:11:41.543438 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.543434    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-metrics-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:11:41.543438 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.543444    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/108dedb2-6c6e-4720-9be9-1ecda5457a65-openshift-service-ca\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:11:41.543680 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.543453    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/108dedb2-6c6e-4720-9be9-1ecda5457a65-maas-api-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:11:41.589498 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.589466    2582 generic.go:356] "Generic (PLEG): container finished" podID="108dedb2-6c6e-4720-9be9-1ecda5457a65" containerID="1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3" exitCode=0
Sep 11 18:11:41.589677 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.589514    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"
Sep 11 18:11:41.589677 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.589520    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89" event={"ID":"108dedb2-6c6e-4720-9be9-1ecda5457a65","Type":"ContainerDied","Data":"1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3"}
Sep 11 18:11:41.589677 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.589646    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89" event={"ID":"108dedb2-6c6e-4720-9be9-1ecda5457a65","Type":"ContainerDied","Data":"5cf29a3d6e9e6f3180f8d1547b735d14ecd988fd8d59dd7b3da8ba58131e5cd7"}
Sep 11 18:11:41.589801 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.589682    2582 scope.go:117] "RemoveContainer" containerID="1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3"
Sep 11 18:11:41.598956 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.598938    2582 scope.go:117] "RemoveContainer" containerID="1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3"
Sep 11 18:11:41.599335 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:11:41.599317    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3\": container with ID starting with 1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3 not found: ID does not exist" containerID="1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3"
Sep 11 18:11:41.599412 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.599348    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3"} err="failed to get container status \"1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3\": rpc error: code = NotFound desc = could not find container \"1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3\": container with ID starting with 1b448ac53217bdd3c06b5ad7d39d1a5fa8eaace2a60a578a7e4a20e9ebe69ab3 not found: ID does not exist"
Sep 11 18:11:41.611466 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.611433    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"]
Sep 11 18:11:41.615533 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:41.615509    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-64600a-9b6cd9b-zpg89"]
Sep 11 18:11:42.634720 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:11:42.634684    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="108dedb2-6c6e-4720-9be9-1ecda5457a65" path="/var/lib/kubelet/pods/108dedb2-6c6e-4720-9be9-1ecda5457a65/volumes"
Sep 11 18:12:41.227118 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.227060    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"]
Sep 11 18:12:41.227719 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.227300    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5" podUID="9b68fb55-3665-47e4-a479-611f89437e11" containerName="payload-processing" containerID="cri-o://b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b" gracePeriod=30
Sep 11 18:12:41.242511 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.242476    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"]
Sep 11 18:12:41.242791 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.242737    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2" podUID="a21fe026-d4d4-4de8-b515-bb60bb173b3b" containerName="payload-pre-processing" containerID="cri-o://faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095" gracePeriod=30
Sep 11 18:12:41.489981 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.489909    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:12:41.523429 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.523401    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:12:41.601751 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.601714    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4jxw\" (UniqueName: \"kubernetes.io/projected/9b68fb55-3665-47e4-a479-611f89437e11-kube-api-access-l4jxw\") pod \"9b68fb55-3665-47e4-a479-611f89437e11\" (UID: \"9b68fb55-3665-47e4-a479-611f89437e11\") "
Sep 11 18:12:41.601924 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.601779    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a21fe026-d4d4-4de8-b515-bb60bb173b3b-plugins-config-volume\") pod \"a21fe026-d4d4-4de8-b515-bb60bb173b3b\" (UID: \"a21fe026-d4d4-4de8-b515-bb60bb173b3b\") "
Sep 11 18:12:41.601924 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.601867    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngsxw\" (UniqueName: \"kubernetes.io/projected/a21fe026-d4d4-4de8-b515-bb60bb173b3b-kube-api-access-ngsxw\") pod \"a21fe026-d4d4-4de8-b515-bb60bb173b3b\" (UID: \"a21fe026-d4d4-4de8-b515-bb60bb173b3b\") "
Sep 11 18:12:41.601924 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.601917    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b68fb55-3665-47e4-a479-611f89437e11-plugins-config-volume\") pod \"9b68fb55-3665-47e4-a479-611f89437e11\" (UID: \"9b68fb55-3665-47e4-a479-611f89437e11\") "
Sep 11 18:12:41.602345 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.602114    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a21fe026-d4d4-4de8-b515-bb60bb173b3b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a21fe026-d4d4-4de8-b515-bb60bb173b3b" (UID: "a21fe026-d4d4-4de8-b515-bb60bb173b3b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:12:41.602345 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.602319    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b68fb55-3665-47e4-a479-611f89437e11-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b68fb55-3665-47e4-a479-611f89437e11" (UID: "9b68fb55-3665-47e4-a479-611f89437e11"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:12:41.602442 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.602336    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a21fe026-d4d4-4de8-b515-bb60bb173b3b-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:12:41.604050 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.604017    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b68fb55-3665-47e4-a479-611f89437e11-kube-api-access-l4jxw" (OuterVolumeSpecName: "kube-api-access-l4jxw") pod "9b68fb55-3665-47e4-a479-611f89437e11" (UID: "9b68fb55-3665-47e4-a479-611f89437e11"). InnerVolumeSpecName "kube-api-access-l4jxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:12:41.604175 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.604133    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a21fe026-d4d4-4de8-b515-bb60bb173b3b-kube-api-access-ngsxw" (OuterVolumeSpecName: "kube-api-access-ngsxw") pod "a21fe026-d4d4-4de8-b515-bb60bb173b3b" (UID: "a21fe026-d4d4-4de8-b515-bb60bb173b3b"). InnerVolumeSpecName "kube-api-access-ngsxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:12:41.703237 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.703199    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngsxw\" (UniqueName: \"kubernetes.io/projected/a21fe026-d4d4-4de8-b515-bb60bb173b3b-kube-api-access-ngsxw\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:12:41.703237 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.703234    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b68fb55-3665-47e4-a479-611f89437e11-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:12:41.703237 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.703247    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4jxw\" (UniqueName: \"kubernetes.io/projected/9b68fb55-3665-47e4-a479-611f89437e11-kube-api-access-l4jxw\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:12:41.885158 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.885045    2582 generic.go:356] "Generic (PLEG): container finished" podID="a21fe026-d4d4-4de8-b515-bb60bb173b3b" containerID="faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095" exitCode=0
Sep 11 18:12:41.885158 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.885108    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"
Sep 11 18:12:41.885158 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.885123    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2" event={"ID":"a21fe026-d4d4-4de8-b515-bb60bb173b3b","Type":"ContainerDied","Data":"faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095"}
Sep 11 18:12:41.885447 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.885175    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2" event={"ID":"a21fe026-d4d4-4de8-b515-bb60bb173b3b","Type":"ContainerDied","Data":"ea974925eb7fde6331457e7f5c139db828dc8af9ecdcb17bbff03607a4399e1c"}
Sep 11 18:12:41.885447 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.885196    2582 scope.go:117] "RemoveContainer" containerID="faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095"
Sep 11 18:12:41.886668 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.886644    2582 generic.go:356] "Generic (PLEG): container finished" podID="9b68fb55-3665-47e4-a479-611f89437e11" containerID="b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b" exitCode=0
Sep 11 18:12:41.886790 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.886691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5" event={"ID":"9b68fb55-3665-47e4-a479-611f89437e11","Type":"ContainerDied","Data":"b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b"}
Sep 11 18:12:41.886790 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.886741    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5" event={"ID":"9b68fb55-3665-47e4-a479-611f89437e11","Type":"ContainerDied","Data":"a50da8df8afe4c51f20d0338e618815b90a57d2c2af0d6172f5c9b2b52f76ea6"}
Sep 11 18:12:41.886790 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.886699    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"
Sep 11 18:12:41.895000 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.894936    2582 scope.go:117] "RemoveContainer" containerID="faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095"
Sep 11 18:12:41.895336 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:12:41.895316    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095\": container with ID starting with faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095 not found: ID does not exist" containerID="faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095"
Sep 11 18:12:41.895397 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.895346    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095"} err="failed to get container status \"faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095\": rpc error: code = NotFound desc = could not find container \"faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095\": container with ID starting with faf79058253a62591a71c67b8498cf2a406241646aee7c1cb02a4d21008c8095 not found: ID does not exist"
Sep 11 18:12:41.895397 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.895363    2582 scope.go:117] "RemoveContainer" containerID="b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b"
Sep 11 18:12:41.903263 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.903246    2582 scope.go:117] "RemoveContainer" containerID="b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b"
Sep 11 18:12:41.903537 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:12:41.903518    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b\": container with ID starting with b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b not found: ID does not exist" containerID="b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b"
Sep 11 18:12:41.903583 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.903544    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b"} err="failed to get container status \"b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b\": rpc error: code = NotFound desc = could not find container \"b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b\": container with ID starting with b9b76f705a2b00dc90575dbe44839bcb50a6612ff4aa5f04511adf623f142a0b not found: ID does not exist"
Sep 11 18:12:41.912000 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.911978    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"]
Sep 11 18:12:41.916518 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.916495    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f8f6a0-6dfd9bc557-m8cf5"]
Sep 11 18:12:41.927603 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.927578    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"]
Sep 11 18:12:41.935265 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:41.935242    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f8f6a0-79c8b7c7f4-qrjx2"]
Sep 11 18:12:42.635236 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:42.635161    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b68fb55-3665-47e4-a479-611f89437e11" path="/var/lib/kubelet/pods/9b68fb55-3665-47e4-a479-611f89437e11/volumes"
Sep 11 18:12:42.635587 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:42.635524    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a21fe026-d4d4-4de8-b515-bb60bb173b3b" path="/var/lib/kubelet/pods/a21fe026-d4d4-4de8-b515-bb60bb173b3b/volumes"
Sep 11 18:12:59.549339 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.549305    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl"]
Sep 11 18:12:59.552822 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.552796    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-a18ac6-29819160-rs5sl"]
Sep 11 18:12:59.585243 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.585207    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"]
Sep 11 18:12:59.585499 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.585467    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4" podUID="303455fb-866d-40f3-a5c3-4d0e739699af" containerName="payload-processing" containerID="cri-o://e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce" gracePeriod=30
Sep 11 18:12:59.590936 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.590906    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"]
Sep 11 18:12:59.591169 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.591143    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h" podUID="70e673b3-7021-4e59-88ee-3027ee45d1ba" containerName="payload-pre-processing" containerID="cri-o://15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e" gracePeriod=30
Sep 11 18:12:59.867135 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.867112    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 18:12:59.870034 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.870014    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 18:12:59.972296 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.972267    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70e673b3-7021-4e59-88ee-3027ee45d1ba-plugins-config-volume\") pod \"70e673b3-7021-4e59-88ee-3027ee45d1ba\" (UID: \"70e673b3-7021-4e59-88ee-3027ee45d1ba\") "
Sep 11 18:12:59.972514 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.972338    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/303455fb-866d-40f3-a5c3-4d0e739699af-plugins-config-volume\") pod \"303455fb-866d-40f3-a5c3-4d0e739699af\" (UID: \"303455fb-866d-40f3-a5c3-4d0e739699af\") "
Sep 11 18:12:59.972514 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.972392    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-692jm\" (UniqueName: \"kubernetes.io/projected/70e673b3-7021-4e59-88ee-3027ee45d1ba-kube-api-access-692jm\") pod \"70e673b3-7021-4e59-88ee-3027ee45d1ba\" (UID: \"70e673b3-7021-4e59-88ee-3027ee45d1ba\") "
Sep 11 18:12:59.972514 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.972454    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tscj\" (UniqueName: \"kubernetes.io/projected/303455fb-866d-40f3-a5c3-4d0e739699af-kube-api-access-2tscj\") pod \"303455fb-866d-40f3-a5c3-4d0e739699af\" (UID: \"303455fb-866d-40f3-a5c3-4d0e739699af\") "
Sep 11 18:12:59.972806 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.972772    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/303455fb-866d-40f3-a5c3-4d0e739699af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "303455fb-866d-40f3-a5c3-4d0e739699af" (UID: "303455fb-866d-40f3-a5c3-4d0e739699af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:12:59.972899 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.972776    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70e673b3-7021-4e59-88ee-3027ee45d1ba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "70e673b3-7021-4e59-88ee-3027ee45d1ba" (UID: "70e673b3-7021-4e59-88ee-3027ee45d1ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:12:59.974856 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.974830    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/303455fb-866d-40f3-a5c3-4d0e739699af-kube-api-access-2tscj" (OuterVolumeSpecName: "kube-api-access-2tscj") pod "303455fb-866d-40f3-a5c3-4d0e739699af" (UID: "303455fb-866d-40f3-a5c3-4d0e739699af"). InnerVolumeSpecName "kube-api-access-2tscj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:12:59.974990 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.974892    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70e673b3-7021-4e59-88ee-3027ee45d1ba-kube-api-access-692jm" (OuterVolumeSpecName: "kube-api-access-692jm") pod "70e673b3-7021-4e59-88ee-3027ee45d1ba" (UID: "70e673b3-7021-4e59-88ee-3027ee45d1ba"). InnerVolumeSpecName "kube-api-access-692jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:12:59.976038 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.976014    2582 generic.go:356] "Generic (PLEG): container finished" podID="303455fb-866d-40f3-a5c3-4d0e739699af" containerID="e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce" exitCode=0
Sep 11 18:12:59.976170 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.976059    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"
Sep 11 18:12:59.976170 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.976054    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4" event={"ID":"303455fb-866d-40f3-a5c3-4d0e739699af","Type":"ContainerDied","Data":"e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce"}
Sep 11 18:12:59.976170 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.976165    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4" event={"ID":"303455fb-866d-40f3-a5c3-4d0e739699af","Type":"ContainerDied","Data":"630ea6a85c35a58338f0d0694aae517f6265a00f15c7ebb0613c35826dce603b"}
Sep 11 18:12:59.976336 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.976197    2582 scope.go:117] "RemoveContainer" containerID="e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce"
Sep 11 18:12:59.977502 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.977481    2582 generic.go:356] "Generic (PLEG): container finished" podID="70e673b3-7021-4e59-88ee-3027ee45d1ba" containerID="15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e" exitCode=0
Sep 11 18:12:59.977562 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.977517    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"
Sep 11 18:12:59.977562 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.977530    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h" event={"ID":"70e673b3-7021-4e59-88ee-3027ee45d1ba","Type":"ContainerDied","Data":"15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e"}
Sep 11 18:12:59.977641 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.977571    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h" event={"ID":"70e673b3-7021-4e59-88ee-3027ee45d1ba","Type":"ContainerDied","Data":"377b9e983f263a21db46102fb5af83177f66b1c2d78be54fc96ac0ec9ce40210"}
Sep 11 18:12:59.985579 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.985561    2582 scope.go:117] "RemoveContainer" containerID="e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce"
Sep 11 18:12:59.985907 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:12:59.985886    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce\": container with ID starting with e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce not found: ID does not exist" containerID="e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce"
Sep 11 18:12:59.985993 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.985918    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce"} err="failed to get container status \"e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce\": rpc error: code = NotFound desc = could not find container \"e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce\": container with ID starting with e378103cd7b8c19c5b35347a9be1c08f751f32ad28e662c6bae1dc2a1d4480ce not found: ID does not exist"
Sep 11 18:12:59.985993 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.985942    2582 scope.go:117] "RemoveContainer" containerID="15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e"
Sep 11 18:12:59.994142 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.994127    2582 scope.go:117] "RemoveContainer" containerID="15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e"
Sep 11 18:12:59.994409 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:12:59.994388    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e\": container with ID starting with 15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e not found: ID does not exist" containerID="15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e"
Sep 11 18:12:59.994485 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:12:59.994415    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e"} err="failed to get container status \"15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e\": rpc error: code = NotFound desc = could not find container \"15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e\": container with ID starting with 15e6e0e6758dc6a7c64d658fe47bdd278f5fab428ab16119ad101da1242fee5e not found: ID does not exist"
Sep 11 18:13:00.001261 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.001235    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"]
Sep 11 18:13:00.005420 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.005396    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a18ac6-5888b9c6b6-hb48h"]
Sep 11 18:13:00.019845 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.019804    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"]
Sep 11 18:13:00.022775 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.022747    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a18ac6-87759c959-wt2f4"]
Sep 11 18:13:00.073934 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.073894    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tscj\" (UniqueName: \"kubernetes.io/projected/303455fb-866d-40f3-a5c3-4d0e739699af-kube-api-access-2tscj\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:13:00.073934 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.073926    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70e673b3-7021-4e59-88ee-3027ee45d1ba-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:13:00.073934 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.073938    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/303455fb-866d-40f3-a5c3-4d0e739699af-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:13:00.074206 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.073947    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-692jm\" (UniqueName: \"kubernetes.io/projected/70e673b3-7021-4e59-88ee-3027ee45d1ba-kube-api-access-692jm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:13:00.635133 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.635073    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="303455fb-866d-40f3-a5c3-4d0e739699af" path="/var/lib/kubelet/pods/303455fb-866d-40f3-a5c3-4d0e739699af/volumes"
Sep 11 18:13:00.635469 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.635391    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6eadc19f-c6dc-4558-88bd-5d0864859e13" path="/var/lib/kubelet/pods/6eadc19f-c6dc-4558-88bd-5d0864859e13/volumes"
Sep 11 18:13:00.635665 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:00.635652    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70e673b3-7021-4e59-88ee-3027ee45d1ba" path="/var/lib/kubelet/pods/70e673b3-7021-4e59-88ee-3027ee45d1ba/volumes"
Sep 11 18:13:17.772333 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:17.772258    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"]
Sep 11 18:13:17.772856 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:17.772477    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v" podUID="3adfe395-74f6-4d42-b0d0-a6179dcc38eb" containerName="payload-processing" containerID="cri-o://9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03" gracePeriod=30
Sep 11 18:13:17.777333 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:17.777306    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"]
Sep 11 18:13:17.777565 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:17.777525    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n" podUID="ab88d592-d281-41d0-b481-50b5c762e493" containerName="payload-pre-processing" containerID="cri-o://639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae" gracePeriod=30
Sep 11 18:13:18.024931 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.024862    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 18:13:18.056453 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.056425    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 18:13:18.060569 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.060522    2582 generic.go:356] "Generic (PLEG): container finished" podID="3adfe395-74f6-4d42-b0d0-a6179dcc38eb" containerID="9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03" exitCode=0
Sep 11 18:13:18.060569 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.060567    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"
Sep 11 18:13:18.060799 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.060589    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v" event={"ID":"3adfe395-74f6-4d42-b0d0-a6179dcc38eb","Type":"ContainerDied","Data":"9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03"}
Sep 11 18:13:18.060799 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.060644    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v" event={"ID":"3adfe395-74f6-4d42-b0d0-a6179dcc38eb","Type":"ContainerDied","Data":"285a664ab2a8d13afd04a06ef1802bfe57d19e5df95260d73d4121f14f6009c2"}
Sep 11 18:13:18.060799 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.060672    2582 scope.go:117] "RemoveContainer" containerID="9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03"
Sep 11 18:13:18.062128 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.062105    2582 generic.go:356] "Generic (PLEG): container finished" podID="ab88d592-d281-41d0-b481-50b5c762e493" containerID="639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae" exitCode=0
Sep 11 18:13:18.062248 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.062139    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n" event={"ID":"ab88d592-d281-41d0-b481-50b5c762e493","Type":"ContainerDied","Data":"639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae"}
Sep 11 18:13:18.062248 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.062156    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"
Sep 11 18:13:18.062248 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.062190    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n" event={"ID":"ab88d592-d281-41d0-b481-50b5c762e493","Type":"ContainerDied","Data":"0c72c9b0b449c1b748b250ae08f3942eae5d1136dcc56b694114dd2ced46fdf5"}
Sep 11 18:13:18.069313 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.069291    2582 scope.go:117] "RemoveContainer" containerID="9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03"
Sep 11 18:13:18.069602 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:13:18.069583    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03\": container with ID starting with 9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03 not found: ID does not exist" containerID="9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03"
Sep 11 18:13:18.069678 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.069609    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03"} err="failed to get container status \"9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03\": rpc error: code = NotFound desc = could not find container \"9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03\": container with ID starting with 9c249112fa750e01d978b78e7f3a10c403de692afac8ecd5a048e171052a1e03 not found: ID does not exist"
Sep 11 18:13:18.069678 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.069625    2582 scope.go:117] "RemoveContainer" containerID="639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae"
Sep 11 18:13:18.077595 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.077548    2582 scope.go:117] "RemoveContainer" containerID="639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae"
Sep 11 18:13:18.077899 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:13:18.077875    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae\": container with ID starting with 639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae not found: ID does not exist" containerID="639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae"
Sep 11 18:13:18.078065 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.077910    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae"} err="failed to get container status \"639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae\": rpc error: code = NotFound desc = could not find container \"639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae\": container with ID starting with 639baa71f30be25acfb738220be49b943f6d131915928e72347a18f060f5abae not found: ID does not exist"
Sep 11 18:13:18.133797 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.133753    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lk7f9\" (UniqueName: \"kubernetes.io/projected/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-kube-api-access-lk7f9\") pod \"3adfe395-74f6-4d42-b0d0-a6179dcc38eb\" (UID: \"3adfe395-74f6-4d42-b0d0-a6179dcc38eb\") "
Sep 11 18:13:18.133797 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.133796    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab88d592-d281-41d0-b481-50b5c762e493-plugins-config-volume\") pod \"ab88d592-d281-41d0-b481-50b5c762e493\" (UID: \"ab88d592-d281-41d0-b481-50b5c762e493\") "
Sep 11 18:13:18.134067 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.133824    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-plugins-config-volume\") pod \"3adfe395-74f6-4d42-b0d0-a6179dcc38eb\" (UID: \"3adfe395-74f6-4d42-b0d0-a6179dcc38eb\") "
Sep 11 18:13:18.134067 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.133941    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjshj\" (UniqueName: \"kubernetes.io/projected/ab88d592-d281-41d0-b481-50b5c762e493-kube-api-access-xjshj\") pod \"ab88d592-d281-41d0-b481-50b5c762e493\" (UID: \"ab88d592-d281-41d0-b481-50b5c762e493\") "
Sep 11 18:13:18.134261 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.134239    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3adfe395-74f6-4d42-b0d0-a6179dcc38eb" (UID: "3adfe395-74f6-4d42-b0d0-a6179dcc38eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:18.134261 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.134241    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab88d592-d281-41d0-b481-50b5c762e493-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab88d592-d281-41d0-b481-50b5c762e493" (UID: "ab88d592-d281-41d0-b481-50b5c762e493"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:18.136206 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.136182    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-kube-api-access-lk7f9" (OuterVolumeSpecName: "kube-api-access-lk7f9") pod "3adfe395-74f6-4d42-b0d0-a6179dcc38eb" (UID: "3adfe395-74f6-4d42-b0d0-a6179dcc38eb"). InnerVolumeSpecName "kube-api-access-lk7f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:13:18.136276 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.136219    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab88d592-d281-41d0-b481-50b5c762e493-kube-api-access-xjshj" (OuterVolumeSpecName: "kube-api-access-xjshj") pod "ab88d592-d281-41d0-b481-50b5c762e493" (UID: "ab88d592-d281-41d0-b481-50b5c762e493"). InnerVolumeSpecName "kube-api-access-xjshj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:13:18.235273 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.235230    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lk7f9\" (UniqueName: \"kubernetes.io/projected/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-kube-api-access-lk7f9\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:13:18.235273 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.235263    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab88d592-d281-41d0-b481-50b5c762e493-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:13:18.235273 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.235273    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3adfe395-74f6-4d42-b0d0-a6179dcc38eb-plugins-config-volume\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:13:18.235273 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.235283    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjshj\" (UniqueName: \"kubernetes.io/projected/ab88d592-d281-41d0-b481-50b5c762e493-kube-api-access-xjshj\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:13:18.411945 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.411911    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"]
Sep 11 18:13:18.425229 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.425201    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1bf937-75cc88878-vnw5n"]
Sep 11 18:13:18.458933 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.458899    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"]
Sep 11 18:13:18.464708 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.464678    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1bf937-787c9b9c57-wp72v"]
Sep 11 18:13:18.633406 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.633312    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3adfe395-74f6-4d42-b0d0-a6179dcc38eb" path="/var/lib/kubelet/pods/3adfe395-74f6-4d42-b0d0-a6179dcc38eb/volumes"
Sep 11 18:13:18.633777 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:18.633757    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab88d592-d281-41d0-b481-50b5c762e493" path="/var/lib/kubelet/pods/ab88d592-d281-41d0-b481-50b5c762e493/volumes"
Sep 11 18:13:52.961771 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:13:52.961733    2582 scope.go:117] "RemoveContainer" containerID="4945dfe2ede720006d9d2691963f4a0b109e5d19e9fdd78f4d2e0b84be2e3a5a"
Sep 11 18:18:09.687354 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687321    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"]
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687644    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="303455fb-866d-40f3-a5c3-4d0e739699af" containerName="payload-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687656    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="303455fb-866d-40f3-a5c3-4d0e739699af" containerName="payload-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687669    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a21fe026-d4d4-4de8-b515-bb60bb173b3b" containerName="payload-pre-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687675    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a21fe026-d4d4-4de8-b515-bb60bb173b3b" containerName="payload-pre-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687681    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b68fb55-3665-47e4-a479-611f89437e11" containerName="payload-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687687    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b68fb55-3665-47e4-a479-611f89437e11" containerName="payload-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687692    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1fa9d81-8da0-4d34-95f6-6d8db1e79178" containerName="maas-api"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687697    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1fa9d81-8da0-4d34-95f6-6d8db1e79178" containerName="maas-api"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687704    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3adfe395-74f6-4d42-b0d0-a6179dcc38eb" containerName="payload-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687709    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3adfe395-74f6-4d42-b0d0-a6179dcc38eb" containerName="payload-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687723    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70e673b3-7021-4e59-88ee-3027ee45d1ba" containerName="payload-pre-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687728    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e673b3-7021-4e59-88ee-3027ee45d1ba" containerName="payload-pre-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687738    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="108dedb2-6c6e-4720-9be9-1ecda5457a65" containerName="maas-api"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687743    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="108dedb2-6c6e-4720-9be9-1ecda5457a65" containerName="maas-api"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687750    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab88d592-d281-41d0-b481-50b5c762e493" containerName="payload-pre-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687756    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab88d592-d281-41d0-b481-50b5c762e493" containerName="payload-pre-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687806    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3adfe395-74f6-4d42-b0d0-a6179dcc38eb" containerName="payload-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687814    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a21fe026-d4d4-4de8-b515-bb60bb173b3b" containerName="payload-pre-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687821    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b68fb55-3665-47e4-a479-611f89437e11" containerName="payload-processing"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687827    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1fa9d81-8da0-4d34-95f6-6d8db1e79178" containerName="maas-api"
Sep 11 18:18:09.687823 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687833    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="108dedb2-6c6e-4720-9be9-1ecda5457a65" containerName="maas-api"
Sep 11 18:18:09.688480 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687840    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="70e673b3-7021-4e59-88ee-3027ee45d1ba" containerName="payload-pre-processing"
Sep 11 18:18:09.688480 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687846    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="303455fb-866d-40f3-a5c3-4d0e739699af" containerName="payload-processing"
Sep 11 18:18:09.688480 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.687852    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab88d592-d281-41d0-b481-50b5c762e493" containerName="payload-pre-processing"
Sep 11 18:18:09.690961 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.690943    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.693961 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.693930    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-c0ed78ba-kserve-self-signed-certs\""
Sep 11 18:18:09.707626 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.707599    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"]
Sep 11 18:18:09.858529 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.858491    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-home\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.858529 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.858533    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfgx4\" (UniqueName: \"kubernetes.io/projected/3cdba203-7fc8-43a7-b022-c8978ff57d72-kube-api-access-hfgx4\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.858758 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.858587    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-dshm\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.858758 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.858681    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-tmp-dir\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.858758 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.858713    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-model-cache\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.858758 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.858749    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cdba203-7fc8-43a7-b022-c8978ff57d72-tls-certs\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.959847 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.959753    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-dshm\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.959847 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.959822    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-tmp-dir\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.959847 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.959840    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-model-cache\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.960160 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.959870    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cdba203-7fc8-43a7-b022-c8978ff57d72-tls-certs\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.960160 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.959894    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-home\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.960160 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.959923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfgx4\" (UniqueName: \"kubernetes.io/projected/3cdba203-7fc8-43a7-b022-c8978ff57d72-kube-api-access-hfgx4\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.960326 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.960303    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-tmp-dir\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.960390 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.960376    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-home\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.960438 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.960376    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-model-cache\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.962303 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.962276    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-dshm\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.962472 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.962453    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cdba203-7fc8-43a7-b022-c8978ff57d72-tls-certs\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:09.970068 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:09.970043    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfgx4\" (UniqueName: \"kubernetes.io/projected/3cdba203-7fc8-43a7-b022-c8978ff57d72-kube-api-access-hfgx4\") pod \"e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") " pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:10.000359 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:10.000319    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:10.281198 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:10.281105    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"]
Sep 11 18:18:10.285023 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:18:10.284992    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3cdba203_7fc8_43a7_b022_c8978ff57d72.slice/crio-0e6cb177a3d2e5325f4148b2c82f92cc957c4e7cb47eb795ccee02b19b460863 WatchSource:0}: Error finding container 0e6cb177a3d2e5325f4148b2c82f92cc957c4e7cb47eb795ccee02b19b460863: Status 404 returned error can't find the container with id 0e6cb177a3d2e5325f4148b2c82f92cc957c4e7cb47eb795ccee02b19b460863
Sep 11 18:18:10.401656 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:10.401616    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw" event={"ID":"3cdba203-7fc8-43a7-b022-c8978ff57d72","Type":"ContainerStarted","Data":"3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31"}
Sep 11 18:18:10.401656 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:10.401663    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw" event={"ID":"3cdba203-7fc8-43a7-b022-c8978ff57d72","Type":"ContainerStarted","Data":"0e6cb177a3d2e5325f4148b2c82f92cc957c4e7cb47eb795ccee02b19b460863"}
Sep 11 18:18:10.425639 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:10.425569    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw" podStartSLOduration=1.425541699 podStartE2EDuration="1.425541699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:10.424841633 +0000 UTC m=+2358.313019980" watchObservedRunningTime="2026-09-11 18:18:10.425541699 +0000 UTC m=+2358.313720044"
Sep 11 18:18:20.001067 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:20.001029    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:20.001461 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:20.001097    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:20.013409 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:20.013385    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:20.459285 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:20.459254    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:25.917765 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:25.917731    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"]
Sep 11 18:18:40.937594 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:40.937553    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw" podUID="3cdba203-7fc8-43a7-b022-c8978ff57d72" containerName="main" containerID="cri-o://3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31" gracePeriod=45
Sep 11 18:18:41.188969 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.188911    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:41.335678 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.335643    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-model-cache\") pod \"3cdba203-7fc8-43a7-b022-c8978ff57d72\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") "
Sep 11 18:18:41.335860 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.335702    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-dshm\") pod \"3cdba203-7fc8-43a7-b022-c8978ff57d72\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") "
Sep 11 18:18:41.335860 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.335736    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cdba203-7fc8-43a7-b022-c8978ff57d72-tls-certs\") pod \"3cdba203-7fc8-43a7-b022-c8978ff57d72\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") "
Sep 11 18:18:41.335860 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.335774    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfgx4\" (UniqueName: \"kubernetes.io/projected/3cdba203-7fc8-43a7-b022-c8978ff57d72-kube-api-access-hfgx4\") pod \"3cdba203-7fc8-43a7-b022-c8978ff57d72\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") "
Sep 11 18:18:41.335860 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.335809    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-tmp-dir\") pod \"3cdba203-7fc8-43a7-b022-c8978ff57d72\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") "
Sep 11 18:18:41.335860 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.335853    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-home\") pod \"3cdba203-7fc8-43a7-b022-c8978ff57d72\" (UID: \"3cdba203-7fc8-43a7-b022-c8978ff57d72\") "
Sep 11 18:18:41.336177 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.335999    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-model-cache" (OuterVolumeSpecName: "model-cache") pod "3cdba203-7fc8-43a7-b022-c8978ff57d72" (UID: "3cdba203-7fc8-43a7-b022-c8978ff57d72"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:18:41.336177 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.336151    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3cdba203-7fc8-43a7-b022-c8978ff57d72" (UID: "3cdba203-7fc8-43a7-b022-c8978ff57d72"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:18:41.336264 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.336220    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-home" (OuterVolumeSpecName: "home") pod "3cdba203-7fc8-43a7-b022-c8978ff57d72" (UID: "3cdba203-7fc8-43a7-b022-c8978ff57d72"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:18:41.336309 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.336283    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-tmp-dir\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:18:41.336309 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.336303    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-model-cache\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:18:41.338350 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.338324    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cdba203-7fc8-43a7-b022-c8978ff57d72-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3cdba203-7fc8-43a7-b022-c8978ff57d72" (UID: "3cdba203-7fc8-43a7-b022-c8978ff57d72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:18:41.338350 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.338329    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-dshm" (OuterVolumeSpecName: "dshm") pod "3cdba203-7fc8-43a7-b022-c8978ff57d72" (UID: "3cdba203-7fc8-43a7-b022-c8978ff57d72"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:18:41.338511 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.338355    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cdba203-7fc8-43a7-b022-c8978ff57d72-kube-api-access-hfgx4" (OuterVolumeSpecName: "kube-api-access-hfgx4") pod "3cdba203-7fc8-43a7-b022-c8978ff57d72" (UID: "3cdba203-7fc8-43a7-b022-c8978ff57d72"). InnerVolumeSpecName "kube-api-access-hfgx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:18:41.437789 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.437748    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-dshm\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:18:41.437789 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.437782    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cdba203-7fc8-43a7-b022-c8978ff57d72-tls-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:18:41.438005 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.437800    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfgx4\" (UniqueName: \"kubernetes.io/projected/3cdba203-7fc8-43a7-b022-c8978ff57d72-kube-api-access-hfgx4\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:18:41.438005 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.437812    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cdba203-7fc8-43a7-b022-c8978ff57d72-home\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:18:41.556266 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.556166    2582 generic.go:356] "Generic (PLEG): container finished" podID="3cdba203-7fc8-43a7-b022-c8978ff57d72" containerID="3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31" exitCode=0
Sep 11 18:18:41.556266 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.556222    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"
Sep 11 18:18:41.556266 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.556231    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw" event={"ID":"3cdba203-7fc8-43a7-b022-c8978ff57d72","Type":"ContainerDied","Data":"3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31"}
Sep 11 18:18:41.556266 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.556263    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw" event={"ID":"3cdba203-7fc8-43a7-b022-c8978ff57d72","Type":"ContainerDied","Data":"0e6cb177a3d2e5325f4148b2c82f92cc957c4e7cb47eb795ccee02b19b460863"}
Sep 11 18:18:41.556592 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.556280    2582 scope.go:117] "RemoveContainer" containerID="3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31"
Sep 11 18:18:41.565618 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.565597    2582 scope.go:117] "RemoveContainer" containerID="3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31"
Sep 11 18:18:41.565939 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:18:41.565920    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31\": container with ID starting with 3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31 not found: ID does not exist" containerID="3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31"
Sep 11 18:18:41.565988 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.565949    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31"} err="failed to get container status \"3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31\": rpc error: code = NotFound desc = could not find container \"3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31\": container with ID starting with 3dd5c591e283ce1a9e797f5a26951f72b38e7f574d754c9eb3f68a665a5bfe31 not found: ID does not exist"
Sep 11 18:18:41.590838 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.590801    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"]
Sep 11 18:18:41.596553 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:41.596520    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-c0ed78ba-kserve-74b747dcd7-9hlvw"]
Sep 11 18:18:42.634307 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:42.634273    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cdba203-7fc8-43a7-b022-c8978ff57d72" path="/var/lib/kubelet/pods/3cdba203-7fc8-43a7-b022-c8978ff57d72/volumes"
Sep 11 18:18:44.380279 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.380242    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"]
Sep 11 18:18:44.380665 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.380559    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cdba203-7fc8-43a7-b022-c8978ff57d72" containerName="main"
Sep 11 18:18:44.380665 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.380570    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cdba203-7fc8-43a7-b022-c8978ff57d72" containerName="main"
Sep 11 18:18:44.380665 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.380638    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cdba203-7fc8-43a7-b022-c8978ff57d72" containerName="main"
Sep 11 18:18:44.385069 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.385045    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.389722 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.389692    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 18:18:44.389722 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.389702    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 18:18:44.390345 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.390331    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 18:18:44.390417 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.390378    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:18:44.390459 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.390448    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5vrxn\""
Sep 11 18:18:44.403670 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.403637    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"]
Sep 11 18:18:44.465633 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.465590    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-metrics-certs\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.465819 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.465675    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-maas-api-tls\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.465819 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.465704    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-openshift-service-ca\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.465819 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.465736    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg2qk\" (UniqueName: \"kubernetes.io/projected/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-kube-api-access-xg2qk\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.567063 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.567030    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-maas-api-tls\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.567300 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.567107    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-openshift-service-ca\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.567300 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.567161    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg2qk\" (UniqueName: \"kubernetes.io/projected/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-kube-api-access-xg2qk\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.567300 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.567212    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-metrics-certs\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.567999 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.567958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-openshift-service-ca\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.569695 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.569667    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-maas-api-tls\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.569886 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.569868    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-metrics-certs\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.581353 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.581326    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg2qk\" (UniqueName: \"kubernetes.io/projected/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-kube-api-access-xg2qk\") pod \"maas-api-579ffb46f9-vmfm4\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") " pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.697040 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.697002    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:44.974177 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.974152    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"]
Sep 11 18:18:44.976215 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:18:44.976193    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dc49c3d_4c52_4f98_b669_c42fa48ca9cb.slice/crio-c0a2239e0e281f1e7bb2cf7bc8fc565bcdb695149a8a328951d695b605e52e60 WatchSource:0}: Error finding container c0a2239e0e281f1e7bb2cf7bc8fc565bcdb695149a8a328951d695b605e52e60: Status 404 returned error can't find the container with id c0a2239e0e281f1e7bb2cf7bc8fc565bcdb695149a8a328951d695b605e52e60
Sep 11 18:18:44.977448 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:44.977431    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:18:45.578237 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:45.578204    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4" event={"ID":"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb","Type":"ContainerStarted","Data":"231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053"}
Sep 11 18:18:45.578597 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:45.578262    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4" event={"ID":"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb","Type":"ContainerStarted","Data":"c0a2239e0e281f1e7bb2cf7bc8fc565bcdb695149a8a328951d695b605e52e60"}
Sep 11 18:18:45.578597 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:45.578320    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:45.598315 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:45.598265    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4" podStartSLOduration=1.071063907 podStartE2EDuration="1.598247751s" totalImagesPullingTime="527.183844ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:45.596614704 +0000 UTC m=+2393.484793049" watchObservedRunningTime="2026-09-11 18:18:45.598247751 +0000 UTC m=+2393.486426096"
Sep 11 18:18:51.588877 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:51.588848    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:18:52.093220 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.093181    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"]
Sep 11 18:18:52.096735 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.096713    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.114578 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.114542    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"]
Sep 11 18:18:52.246881 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.246838    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-maas-api-tls\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.247141 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.246899    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thgg5\" (UniqueName: \"kubernetes.io/projected/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-kube-api-access-thgg5\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.247141 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.247019    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-metrics-certs\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.247268 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.247145    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-openshift-service-ca\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.348049 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.347954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-metrics-certs\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.348049 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.348037    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-openshift-service-ca\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.348255 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.348062    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-maas-api-tls\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.348255 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.348145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thgg5\" (UniqueName: \"kubernetes.io/projected/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-kube-api-access-thgg5\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.348742 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.348718    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-openshift-service-ca\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.350727 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.350708    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-maas-api-tls\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.350789 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.350761    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-metrics-certs\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.358363 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.358337    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thgg5\" (UniqueName: \"kubernetes.io/projected/d31a6e17-a02c-4ef7-8f95-93ceb3deb692-kube-api-access-thgg5\") pod \"maas-api-6c48469567-t2hcm\" (UID: \"d31a6e17-a02c-4ef7-8f95-93ceb3deb692\") " pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.407559 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.407522    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:52.685527 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:52.685485    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"]
Sep 11 18:18:52.688867 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:18:52.688831    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31a6e17_a02c_4ef7_8f95_93ceb3deb692.slice/crio-8c0d61b7946986301c2643d5d28593a1ed63d8d1babd556e232a2024aef47486 WatchSource:0}: Error finding container 8c0d61b7946986301c2643d5d28593a1ed63d8d1babd556e232a2024aef47486: Status 404 returned error can't find the container with id 8c0d61b7946986301c2643d5d28593a1ed63d8d1babd556e232a2024aef47486
Sep 11 18:18:53.616677 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:53.616580    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm" event={"ID":"d31a6e17-a02c-4ef7-8f95-93ceb3deb692","Type":"ContainerStarted","Data":"2dd0d7cbd9caf72abca1812430bb6743ceda0bf4755f0fd7dcd81a0379d7cc6c"}
Sep 11 18:18:53.616677 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:53.616648    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm" event={"ID":"d31a6e17-a02c-4ef7-8f95-93ceb3deb692","Type":"ContainerStarted","Data":"8c0d61b7946986301c2643d5d28593a1ed63d8d1babd556e232a2024aef47486"}
Sep 11 18:18:53.616875 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:53.616695    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:53.635864 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:53.635808    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm" podStartSLOduration=1.103725781 podStartE2EDuration="1.635791167s" totalImagesPullingTime="532.065386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:53.635209063 +0000 UTC m=+2401.523387408" watchObservedRunningTime="2026-09-11 18:18:53.635791167 +0000 UTC m=+2401.523969512"
Sep 11 18:18:59.626863 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:59.626833    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6c48469567-t2hcm"
Sep 11 18:18:59.686058 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:59.686016    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"]
Sep 11 18:18:59.686275 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:59.686248    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4" podUID="2dc49c3d-4c52-4f98-b669-c42fa48ca9cb" containerName="maas-api" containerID="cri-o://231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053" gracePeriod=30
Sep 11 18:18:59.937708 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:18:59.937681    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:19:00.011118 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.011066    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-maas-api-tls\") pod \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") "
Sep 11 18:19:00.011118 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.011122    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-openshift-service-ca\") pod \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") "
Sep 11 18:19:00.011375 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.011143    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg2qk\" (UniqueName: \"kubernetes.io/projected/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-kube-api-access-xg2qk\") pod \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") "
Sep 11 18:19:00.011375 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.011209    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-metrics-certs\") pod \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\" (UID: \"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb\") "
Sep 11 18:19:00.011559 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.011443    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2dc49c3d-4c52-4f98-b669-c42fa48ca9cb" (UID: "2dc49c3d-4c52-4f98-b669-c42fa48ca9cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:19:00.013291 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.013267    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2dc49c3d-4c52-4f98-b669-c42fa48ca9cb" (UID: "2dc49c3d-4c52-4f98-b669-c42fa48ca9cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:19:00.013377 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.013329    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2dc49c3d-4c52-4f98-b669-c42fa48ca9cb" (UID: "2dc49c3d-4c52-4f98-b669-c42fa48ca9cb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:19:00.013471 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.013450    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-kube-api-access-xg2qk" (OuterVolumeSpecName: "kube-api-access-xg2qk") pod "2dc49c3d-4c52-4f98-b669-c42fa48ca9cb" (UID: "2dc49c3d-4c52-4f98-b669-c42fa48ca9cb"). InnerVolumeSpecName "kube-api-access-xg2qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:19:00.112267 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.112223    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-maas-api-tls\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:19:00.112267 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.112256    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-openshift-service-ca\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:19:00.112267 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.112273    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg2qk\" (UniqueName: \"kubernetes.io/projected/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-kube-api-access-xg2qk\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:19:00.112516 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.112284    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb-metrics-certs\") on node \"ip-10-0-140-215.ec2.internal\" DevicePath \"\""
Sep 11 18:19:00.648799 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.648766    2582 generic.go:356] "Generic (PLEG): container finished" podID="2dc49c3d-4c52-4f98-b669-c42fa48ca9cb" containerID="231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053" exitCode=0
Sep 11 18:19:00.649262 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.648811    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"
Sep 11 18:19:00.649262 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.648847    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4" event={"ID":"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb","Type":"ContainerDied","Data":"231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053"}
Sep 11 18:19:00.649262 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.648912    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4" event={"ID":"2dc49c3d-4c52-4f98-b669-c42fa48ca9cb","Type":"ContainerDied","Data":"c0a2239e0e281f1e7bb2cf7bc8fc565bcdb695149a8a328951d695b605e52e60"}
Sep 11 18:19:00.649262 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.648947    2582 scope.go:117] "RemoveContainer" containerID="231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053"
Sep 11 18:19:00.657413 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.657393    2582 scope.go:117] "RemoveContainer" containerID="231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053"
Sep 11 18:19:00.657762 ip-10-0-140-215 kubenswrapper[2582]: E0911 18:19:00.657741    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053\": container with ID starting with 231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053 not found: ID does not exist" containerID="231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053"
Sep 11 18:19:00.657812 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.657772    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053"} err="failed to get container status \"231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053\": rpc error: code = NotFound desc = could not find container \"231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053\": container with ID starting with 231739cf095c655a4fdc7ad013077b7e5fb351804b44145facea64acea056053 not found: ID does not exist"
Sep 11 18:19:00.674602 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.674561    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"]
Sep 11 18:19:00.681638 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:00.681609    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-579ffb46f9-vmfm4"]
Sep 11 18:19:02.641277 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:02.641232    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dc49c3d-4c52-4f98-b669-c42fa48ca9cb" path="/var/lib/kubelet/pods/2dc49c3d-4c52-4f98-b669-c42fa48ca9cb/volumes"
Sep 11 18:19:14.797837 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:14.797764    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d67687f77-p6pjc_7d65f94f-e23d-4dbe-8a5a-397cf0339843/authorino/0.log"
Sep 11 18:19:20.766003 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:20.765963    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nfbc7_d7818d3c-4200-40bd-a528-f4266df97057/manager/0.log"
Sep 11 18:19:21.306988 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:21.306957    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86997d8dbf-58pml_42730934-e078-40e1-b99f-5eddcabf1653/copy-manifests/0.log"
Sep 11 18:19:21.327309 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:21.327267    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86997d8dbf-58pml_42730934-e078-40e1-b99f-5eddcabf1653/manager/0.log"
Sep 11 18:19:22.073688 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:22.073660    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6c48469567-t2hcm_d31a6e17-a02c-4ef7-8f95-93ceb3deb692/maas-api/0.log"
Sep 11 18:19:23.296045 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:23.296014    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d67687f77-p6pjc_7d65f94f-e23d-4dbe-8a5a-397cf0339843/authorino/0.log"
Sep 11 18:19:24.122151 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:24.122119    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-4txbl_bf2615bd-3591-4f82-91c6-728152eab5c0/manager/0.log"
Sep 11 18:19:24.761927 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:24.761895    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8557cb6-c5r7j_ae939b3e-3286-44d3-88dd-6f67a1f79dc6/kube-auth-proxy/0.log"
Sep 11 18:19:25.905382 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:25.905348    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-p99l8_3b5f86cc-32dc-453d-97f0-a16370fe385f/main/0.log"
Sep 11 18:19:26.421002 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:26.420974    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-m9fwc_51cdc09f-d524-47d2-883b-f667124d937d/main/0.log"
Sep 11 18:19:26.544986 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:26.544952    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-zgvnm_8327656a-ad02-4471-bd2a-1a5081be7e3b/main/0.log"
Sep 11 18:19:36.048950 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:36.048915    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cvgh7_46c761dc-4a0b-419e-8c89-86f14b88b4a9/global-pull-secret-syncer/0.log"
Sep 11 18:19:36.157369 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:36.157335    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qccvc_1238055a-bea2-4089-8439-1d0fc275c735/konnectivity-agent/0.log"
Sep 11 18:19:36.302097 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:36.301984    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-215.ec2.internal_1961536c6add67b3e31d5e4dec17d1b8/haproxy/0.log"
Sep 11 18:19:39.952729 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:39.952687    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d67687f77-p6pjc_7d65f94f-e23d-4dbe-8a5a-397cf0339843/authorino/0.log"
Sep 11 18:19:40.361381 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:40.361277    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-4txbl_bf2615bd-3591-4f82-91c6-728152eab5c0/manager/0.log"
Sep 11 18:19:42.721977 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:42.721946    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pqc9d_bd35d437-91ee-45e7-b74e-3860a275e4a3/node-exporter/0.log"
Sep 11 18:19:42.747943 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:42.747909    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pqc9d_bd35d437-91ee-45e7-b74e-3860a275e4a3/kube-rbac-proxy/0.log"
Sep 11 18:19:42.776494 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:42.776466    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pqc9d_bd35d437-91ee-45e7-b74e-3860a275e4a3/init-textfile/0.log"
Sep 11 18:19:42.815905 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:42.815868    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-8z4dg_90ebc72c-0d00-404f-b352-b17a0d410739/kube-rbac-proxy-main/0.log"
Sep 11 18:19:42.847026 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:42.846973    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-8z4dg_90ebc72c-0d00-404f-b352-b17a0d410739/kube-rbac-proxy-self/0.log"
Sep 11 18:19:42.879187 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:42.879156    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-8z4dg_90ebc72c-0d00-404f-b352-b17a0d410739/openshift-state-metrics/0.log"
Sep 11 18:19:42.924310 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:42.924280    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28c5c7f2-9872-4065-b1cf-f80735431b17/prometheus/0.log"
Sep 11 18:19:42.956767 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:42.956732    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28c5c7f2-9872-4065-b1cf-f80735431b17/config-reloader/0.log"
Sep 11 18:19:42.989452 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:42.989365    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28c5c7f2-9872-4065-b1cf-f80735431b17/thanos-sidecar/0.log"
Sep 11 18:19:43.024277 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:43.024249    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28c5c7f2-9872-4065-b1cf-f80735431b17/kube-rbac-proxy-web/0.log"
Sep 11 18:19:43.051723 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:43.051691    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28c5c7f2-9872-4065-b1cf-f80735431b17/kube-rbac-proxy/0.log"
Sep 11 18:19:43.085683 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:43.085655    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28c5c7f2-9872-4065-b1cf-f80735431b17/kube-rbac-proxy-thanos/0.log"
Sep 11 18:19:43.116740 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:43.116710    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_28c5c7f2-9872-4065-b1cf-f80735431b17/init-config-reloader/0.log"
Sep 11 18:19:43.447590 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:43.447546    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7fc48577-vzxsx_9a68b63b-ee60-4ddb-85b9-7129ddd885dc/thanos-query/0.log"
Sep 11 18:19:43.486248 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:43.486212    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7fc48577-vzxsx_9a68b63b-ee60-4ddb-85b9-7129ddd885dc/kube-rbac-proxy-web/0.log"
Sep 11 18:19:43.521146 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:43.521115    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7fc48577-vzxsx_9a68b63b-ee60-4ddb-85b9-7129ddd885dc/kube-rbac-proxy/0.log"
Sep 11 18:19:43.553302 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:43.553270    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7fc48577-vzxsx_9a68b63b-ee60-4ddb-85b9-7129ddd885dc/prom-label-proxy/0.log"
Sep 11 18:19:43.583711 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:43.583682    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7fc48577-vzxsx_9a68b63b-ee60-4ddb-85b9-7129ddd885dc/kube-rbac-proxy-rules/0.log"
Sep 11 18:19:43.623671 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:43.623580    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d7fc48577-vzxsx_9a68b63b-ee60-4ddb-85b9-7129ddd885dc/kube-rbac-proxy-metrics/0.log"
Sep 11 18:19:46.239841 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.239804    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"]
Sep 11 18:19:46.240404 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.240199    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dc49c3d-4c52-4f98-b669-c42fa48ca9cb" containerName="maas-api"
Sep 11 18:19:46.240404 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.240212    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dc49c3d-4c52-4f98-b669-c42fa48ca9cb" containerName="maas-api"
Sep 11 18:19:46.240404 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.240265    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dc49c3d-4c52-4f98-b669-c42fa48ca9cb" containerName="maas-api"
Sep 11 18:19:46.243262 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.243244    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.246066 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.246041    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9jhl8\"/\"default-dockercfg-d7m62\""
Sep 11 18:19:46.246233 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.246210    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9jhl8\"/\"openshift-service-ca.crt\""
Sep 11 18:19:46.247267 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.247253    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9jhl8\"/\"kube-root-ca.crt\""
Sep 11 18:19:46.254363 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.254338    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"]
Sep 11 18:19:46.331381 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.331340    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tc6r\" (UniqueName: \"kubernetes.io/projected/16033b52-f2f4-4b1c-970c-cb3264fcc21f-kube-api-access-2tc6r\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.331381 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.331382    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-sys\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.331646 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.331399    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-proc\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.331646 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.331424    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-lib-modules\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.331646 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.331481    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-podres\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.432940 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.432889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tc6r\" (UniqueName: \"kubernetes.io/projected/16033b52-f2f4-4b1c-970c-cb3264fcc21f-kube-api-access-2tc6r\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.432940 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.432942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-sys\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.432940 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.432959    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-proc\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.433278 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.432985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-lib-modules\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.433278 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.433006    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-podres\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.433278 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.433048    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-sys\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.433278 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.433055    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-proc\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.433278 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.433163    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-podres\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.433278 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.433171    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16033b52-f2f4-4b1c-970c-cb3264fcc21f-lib-modules\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.443179 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.443153    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tc6r\" (UniqueName: \"kubernetes.io/projected/16033b52-f2f4-4b1c-970c-cb3264fcc21f-kube-api-access-2tc6r\") pod \"perf-node-gather-daemonset-prnqc\" (UID: \"16033b52-f2f4-4b1c-970c-cb3264fcc21f\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.492662 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.492570    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-58fffc6dc9-ml9f9_73abb0f1-f5a3-4a5c-8e73-a364a985f22c/console/0.log"
Sep 11 18:19:46.554388 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.554356    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:46.836234 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.836202    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"]
Sep 11 18:19:46.838855 ip-10-0-140-215 kubenswrapper[2582]: W0911 18:19:46.838825    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod16033b52_f2f4_4b1c_970c_cb3264fcc21f.slice/crio-2e15026c9c55264894f77294cbf9cd04c9ae163519d3ca8ded5f06267ec1ad46 WatchSource:0}: Error finding container 2e15026c9c55264894f77294cbf9cd04c9ae163519d3ca8ded5f06267ec1ad46: Status 404 returned error can't find the container with id 2e15026c9c55264894f77294cbf9cd04c9ae163519d3ca8ded5f06267ec1ad46
Sep 11 18:19:46.864795 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:46.864758    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc" event={"ID":"16033b52-f2f4-4b1c-970c-cb3264fcc21f","Type":"ContainerStarted","Data":"2e15026c9c55264894f77294cbf9cd04c9ae163519d3ca8ded5f06267ec1ad46"}
Sep 11 18:19:47.871449 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:47.871402    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc" event={"ID":"16033b52-f2f4-4b1c-970c-cb3264fcc21f","Type":"ContainerStarted","Data":"7ab813e67fabe107df71a5d9e7e76d9de1eec6e879c5e8de7ba53ba4ff332c2a"}
Sep 11 18:19:47.871848 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:47.871521    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:47.892675 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:47.892620    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc" podStartSLOduration=1.8926042779999999 podStartE2EDuration="1.892604278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:19:47.891881272 +0000 UTC m=+2455.780059635" watchObservedRunningTime="2026-09-11 18:19:47.892604278 +0000 UTC m=+2455.780782624"
Sep 11 18:19:48.722174 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:48.722143    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2tj6f_614f25b1-5b49-4f8c-8e90-af062ab8942e/dns/0.log"
Sep 11 18:19:48.755393 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:48.755368    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2tj6f_614f25b1-5b49-4f8c-8e90-af062ab8942e/kube-rbac-proxy/0.log"
Sep 11 18:19:48.900683 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:48.900650    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-88zgk_1ac45d7c-f7d4-422d-8e43-3cbc853fe1df/dns-node-resolver/0.log"
Sep 11 18:19:49.557561 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:49.557527    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-59bbd88f7d-bkxbf_20351bca-486b-456e-83b3-8be534dcbf3a/registry/0.log"
Sep 11 18:19:49.613873 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:49.613840    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkt44_015e5b25-5f9b-4c49-b6ac-575f4065c9ea/node-ca/0.log"
Sep 11 18:19:51.540370 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:51.540336    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8557cb6-c5r7j_ae939b3e-3286-44d3-88dd-6f67a1f79dc6/kube-auth-proxy/0.log"
Sep 11 18:19:52.620983 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:52.620950    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bs2vs_5614e759-35f7-4848-9eb6-f1fa63aba1ba/serve-healthcheck-canary/0.log"
Sep 11 18:19:53.261875 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:53.261848    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xnbll_b16d21f2-b86b-4cb0-9c06-00bc7cb53af0/kube-rbac-proxy/0.log"
Sep 11 18:19:53.285621 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:53.285583    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xnbll_b16d21f2-b86b-4cb0-9c06-00bc7cb53af0/exporter/0.log"
Sep 11 18:19:53.309907 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:53.309817    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xnbll_b16d21f2-b86b-4cb0-9c06-00bc7cb53af0/extractor/0.log"
Sep 11 18:19:53.886828 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:53.886798    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-prnqc"
Sep 11 18:19:56.133347 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:56.133250    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nfbc7_d7818d3c-4200-40bd-a528-f4266df97057/manager/0.log"
Sep 11 18:19:56.581048 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:56.581012    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86997d8dbf-58pml_42730934-e078-40e1-b99f-5eddcabf1653/manager/0.log"
Sep 11 18:19:56.629332 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:56.629306    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86997d8dbf-58pml_42730934-e078-40e1-b99f-5eddcabf1653/copy-manifests/0.log"
Sep 11 18:19:57.396758 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:57.396731    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8557cb6-c5r7j_ae939b3e-3286-44d3-88dd-6f67a1f79dc6/kube-auth-proxy/0.log"
Sep 11 18:19:58.046753 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:58.046662    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6476dc957d-sbsnk_d8d7996b-d427-4bc2-911b-6f72ef3a8921/manager/0.log"
Sep 11 18:19:58.080727 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:58.080692    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-llpns_d8cef1c4-4e1e-4d7b-b655-1cdcd495982f/openshift-lws-operator/0.log"
Sep 11 18:19:59.819334 ip-10-0-140-215 kubenswrapper[2582]: I0911 18:19:59.819289    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8557cb6-c5r7j_ae939b3e-3286-44d3-88dd-6f67a1f79dc6/kube-auth-proxy/0.log"